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ABSTRACT 

This collection of 22 papers is organized by the five 
conference topics. "Part 1: Effects of the Perkins Act on Policy and 
Practice" contains five papers: "Federal Legislation as Rorschach 
Test: Methodological Issues in Assessing the Effects of the 1990 
Perkins Act on States and Localities" (Grubb) ; "Effects of the 1990 
Perkins Act on State Administration and Policy" (Herriage) ; "A New 
Vision for Vocational Education: Assessing Implementation of the 1990 
Perkins Act" (Rosenstock) ; "The Effects of Federal Requirements 
Regarding Articulation between Secondary and Postsecondary Vocational 
Education Programs" (Peters, Jr., Makin) ; and "Views of Assessment of 
the 1990 Perkins Act" (Grimsley) . "Part 2: Funding Issues" consists 
of four papers: "Analyzing Federal Expenditures for Vocational 
Education Program Improvement" (Hoachlander) ; "Return of the Debate: 
Can Federal Policy Improve Vocational Education for Special 
Populations?" (Muraskin) ; "Design for the Congressionally Mandated 
Study of the Formula for Distributing Federal Vocational Education 
Funds to the States" (Barro) ; and "1990 Perkins Act Funding Issues at 
the Postsecondary Level" (Zins) . "Part 3: Student Outcomes" has five 
papers: "Assessing Academic Outcomes in Vocational Education" 
(Sticht); "The Success of School-To-Work Transition" (Stone, III); 
"The Relevance of Vocational Education for Subsequent Employment" 
(Bailey) ; "Occupations and Earnings of Former Vocational Education 
Students: Research Design Issues" (Stevens); and "Vocational 
Education and the American Job Market: An Employer's Perspective" 
(Martin) . The four papers in "Part 4: General and Special 
Populations" are as follows: "Individuals with Special Needs in 
Vocational Education: Considerations for the National Assessment" 
(Phelps); "Recommended Directions: (1) The Effects of Flexibility on 
Delivery of Services to Special Populations and (2) Participatory 
Planning" (Maddy-Bernstein) ; "Principal Issues Regarding Native 
Americans to Be Addressed by the National Assessment of Vocational 
Education" (Slater); and "Special Considerations in Assessing 
Vocational Education in Corrections" (Coffey) . "Part 5: Academic and 
Vocational Issues" contains four papers: "Teacher Preparation, 
Qualifications, and Demand" (Finch); "Performance Standards, 
Accountability, and the Quality of Vocational Education" (Hill) ; 
"Integrating Academic and Vocational Education: Guidelines for 
Assessing a Fuzzy Reform" (Stasz, Grubb); and "Impact of Education 
Reform on Vocational Education" (Strickland). (YLB) 
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Introduction 



The National Assessment of Vocational Education was mandated by Congress in the Carl D. 
Perkins Vocational and Applied Technology Education Act of 1990 (P.L. 101-392). The Office of 
Educational Research and Improvement (OERI) was assigned the task of conducting the National 
Assessment through its Office of Research. The legislation called upon OERI to address a substantial 
number of questions about the status of vocational education in the country and the implementation of 
the Perkins Act. 

One of the first steps in the process of preparing the study was to call together experts in the 
field with the most informed opinions about the various aspects of the mandated study. They were 
asked to use their expertise to address three central questions: 

1. What are the principal issues and research questions to be addressed by the 
assessment in the presenter's particular subject area? 

2. What are the principal data sources (e.g M computerized files, administrative 
records, articles, etc.) that can be used to address these issues and questions? If 
existing data sources are not adequate, what new data could reasonably be 
collected for the assessment? 

3. What problems should be anticipated in conducting research on this subject? 
How can these problems be overcome or mitigated? 

The Design Conference for the National Assessment of Vocational Education was held in 
Washington, D.C., on March 13-15, 1991. It featured a series of panels corresponding to the 
subjects delineated in the legislation. In addition to the formal presentations, there was lively debate 
and discussion among the panel members and other participants in the well-informed audience. These 
presentations and discussions have been used to guide the design of the National Assessment study. 

The papers in this document provided the basis for the conference presentations. They are 
organized by conference topic in the order in which they were presented. The contributing authors 
are identified below: 

Thomas Bailey 
Teachers College 
Columbia University 
New York, New York 

Stephen M. Barro 

SMB Economic Research, Inc. 

Washington, D.C. 
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Independent Consultant 
Washington, D.C. 
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Virginia Department of Education) 

Curtis R. Finch 

Virginia Polytechnic Institute and 

State University 
Blacksburg, Virginia 

Joseph W. Grimsley 
Richmond Community College 
Hamlet, North Carolina 

W. Norton Grubb 

The University of California 

Berkeley, California 

Phyllis A. Herriage 

National Center for Research in Vocational Education 
The University of California 
Berkeley, California 

Paul T. Hill 

The RAND Corporation 
Washington, D.C. 

Gary Hoachlander 
MPR Associates, Inc. 
Berkeley, California 

Carolyn Maddy-Bernstein 
The University of Illinois 
Champaign, Illinois 

Robert L. Martin 

U.S. Chamber of Commerce 

Washington, D.C. 

Lana D. Muraskin 

SMB Economic Research, Inc. 

Washington, D.C. 
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Oklahoma Department of Vocational 

and Technical Education 
Stillwater, Oklahoma 
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Part 1: 

Effects of the Perkins Act on Policy and Practice 



/ 



Federal Legislation as Rorschach Test: 
Methodological Issues in Assessing the Effects of the 1990 
Perkins Act on States and Localities 

W. Norton Grubb* 



In many ways, each piece of federal legislation is a Rorschach test. It has become difficult to 
pass any legislation without the support of several interest groups; however, since interest groups 
rarely agree completely on the purposes of any specific piece of legislation, the result is a hybrid of 
intentions with a multitude of purposes. Then different constituents and interest groups can continue 
to wrangle over what Congress really intended, leading to regulatory battles, implementation 
problems— which are nothing more than the differences between what some beneficiaries want to do 
and what executive agencies want them to do— and (sometimes) further congressional actions. 

This is the case with the 1990 Carl Perkins Act. Congress seems to have intended substantial 
reform in vocational education, but the nature of that reform is not completely clear. Ever since the 
1963 Vocational Education Act, Congress has used federal legislation to pursue two major purposes: 
to focus federal funds on students with special educational needs and to improve vocational education 
(Grubb, 1978). Disagreements over the nature and sufficiency of reforms have led to periodic 
revisions— in the 1968 Amendments, the 1976 Amendments, the Carl Perkins Act of 1983, and now 
in the 1990 Act. 

In the recent changes, the focus on students with special needs is clear enough, since the 
Perkins Act directs federal funds to local districts and postsecm^ary institutions with high 
concentrations of such students. But the nature of program in movement remains murky. In my 
interpretation, program improvement is effectively defined in t?'e Perkins Act (for the first time since 
1963) as the integration of academic and vocational education "he development of tech-prep programs 
and other linkages between secondary and postsecondary pro^a* is, and the development of 
performance measures (a way of improving programs by specif mg expected outputs rather than 
inputs). But all of the elements remain vague in the legislation, and, therefore, they are subject to 
various interpretations. There appears to be a great deal of confusion about how to respond to the 
Perkins Act, and already a battle of sorts appears to be shaping up between those who want to use 
federal funds for "business as usual" and the proponents of reform. 1 



The author is Professor in the Schooi of Education and a site director of the National Center for 
Research in Vocational Education (NCRVE), the University of California, Berkeley. However, this paper 
reflects the opinions of the author only; it has not been reviewed by NCRVE and is not an official 
NCRVE publication. 

'For a preliminary report suggesting that states are not taking the need to change seriously, see 
"Problems in Implementing the Perkins Act— Preliminary Report Concerning State Plans," May 10, 1991, 
issued by a coalition of 31 organizations and summarized in Education Daily, May IS, 1991, p.S. 
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Unlike many pieces of legislation, Congress has taken steps in the Perkins Act to determine 
whether its version of reform— its interpretation of the Rorschach test— prevails over others. The 
Oational Assessment of Vocational Education, required by the Act, is the third in a series stretching 
back to 1978. 3 It represents the institutionalization of congressional effort to see whether its will has 
made any difference to state and local practice. The National Assessment's first obligation, then, is 
to see what has happened in response to the new Act. In this paper, written to support the National 
Assessment's efforts, I address the methodological issues that any such examination faces. I discuss 
the crucial effects of the legislation on 1) state administration and policy, 2) local educational 
practices, 3) the articulation between secondary and postsecondary programs, and 4) the labor market 
outcomes associated with vocational education. In each of these sections I outline the existing 
baseline information about current practice, present some appropriate methodological approaches, and 
outline die particular pitfalls researchers are likely to confront. The paper's fifth section describes 
some general considerations that affect all National Assessment research. 

The National Assessment faces an inescapable ambiguity at the outset: What did Congress 
really intend in enacting the new Perkins Act? The direction of any research and the derivative 
methodological issues depend on what questions one wants to ask and where one wants to go. But in 
this case, the crucial questions are not at all obvious, because Congress has not been especially clear 
in its purposes. In the final section of this paper, I offer my interpretation of what the Perkins Act 
requires, arguing that there is only one real question of any importance and suggesting a direction for 
the National Assessment consistent with that interpretation. Where the first five sections are intended 
to be judicious, balanced, even pedestrian, the final section is intended to be argumentative, 
provocative, and possibly injudicious. But that is, after all, the point of a Rorschach test: The 
interpreter must come to his (or her) own conclusion about an ambiguous pattern, despite ambiguity 
about wha f it really means. 

I. EFFECTS ON STATE ADMINISTRATION POLICY 

The 1990 Perkins Act continues the earlier practice of requiring states to establish a state 
policy, contained in a state plan. In some ways, states have greater power than previously. For 
example, state-level committees of practitioners can decide what performance measures to require, 
and states can interpret, if they want to, the meaning of academic and vocational integration and the 
structure of tech-prep programs. But in other ways, state policy has been circumscribed. The 
funding formulas take most of the discretion away from states for sub-state allocation, and state 
departments of education are left with fewer resources for their own initiatives. 

In determining the effects on state policy, a valuable source of baseline information is 
available from the National Center for Research in Vocational Education (NCRVE). The information 
was obtained through a survey of state policies in all SO states in five different areas (secondary 
vocational education, postsecondary vocational education, Job Training Partnership Act programs, 
welfare-to-work programs, and state economic development activities), undertaken during 1990-91 at 
the RAND Corporation (McDonnell and Zellman, forthcoming). This survey provides information on 
pre-Amendment policies, based on telephone surveys to relevant state officials. 



Alternatively, the current assessment is the sixth in a series including reports in 1938, 1963, and 
1967; see Grubb (1978). 
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An obvious step is for the National Assessment to replicate this research, via telephone 
surveys to state officials responsible for secondary and postsecondary policies. Such a resurvey could 
pose many of the same questions asked in the RAND study, as a way of measuring changes. In 
addition, new questions could focus on the elements of reform that seem most important, including 
each state's interpretation of integration, tech-prep, and performance measures. Such a survey is 
almost surely a better way of obtaining information on state policies than reading state plans, since the 
latter are compliance documents providing little information about what states actually do. However, 
telephone questionnaires are not the ideal way to obtain clear information about state practices (as 
distinct from state policies). In the RAND study it has been extremely difficult to get clear 
information on certain state efforts, and practices (including difficulties and outright errors in 
implementing new legislation) are unlikely to show up in responses to questionnaires. It is therefore 
necessary to develop other sources of information about state responses. Therefore, all investigations 
of local practices and responses to the Perkins Act should incorporate questions about state practices, 
to gain a different perspective on how states have changed. 

Among the special problems this investigation should consider are the following: 

The effects of nonvocational state policies: In the past, state policies which have nothing 
specific to do with vocational education have had tremendous effects on vocational education. The 
most obvious example is the effect of increasing graduation requirements on vocational enrollments 
(Clune, White, and Patterson, 1989). State credentialing requirements, course approval procedures, 
and textbook requirements also can affect the ability of localities to modify their curricula, and have 
already affected local efforts to integrate academic and vocational education (Grubb et al., 1991; 
Bodilly et al., forthcoming). In addition, because several reforms embedded in the Perkins Act 
potentially affect academic as well as vocational education, it would be valuable to know if states 
modify their "academic" policies or provide funding from their general ftinds (as distinct from 
vocational funding) to enhance reforms contained in the Perkins Act. 

Effects on secondary versus postsecondary state policies: The 1990 Perkins Act, like its 
predecessors, states that a single state agency shall receive federal funds. In the past, the majority of 
these agencies have been the state departments of education responsible principally for K-12 
education. They have conveyed funds for postsecondary programs by interagency agreements to the 
state department responsible for community colleges, technical institutes, and area vocational schools. 
Therefore, a complete description of effects on state policies needs to query the state departments 
responsible for postsecondary programs. 

A common complaint of postsecondary institutions is that they have little influence on state 
policy when the single state agency is a K-12 department. Since enrollments in secondary vocational 
education have been decreasing while postsecondary enrollments have been increasing, such an 
imbalance— if true— would be potentially detrimental to having federal funds promote reforms at the 
level where vocational education is increasingly important. Therefore, the National Assessment's 
analysis of state policies should investigate the claims of undue secondary influence on state policy, 
by questions at the state and local levels and by identifying ways in which state policies have been 
difficult for postsecondary institutions to follow. 

Types of performance measures being adopted: The Perkins Act requires state 
committees of practitioners to adopt performance measures by August 1992. It will therefore be 
possible to determine what these measures are in late 1992. However, since the Perkins Act requires 
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only that measures be developed (not implemented) and are silent about how such measures are to be 
used, the real effects of performance measures on data collected, state use of performance measures, 
and reforms undertaken in response to performance measures will not be clear until later, perhaps 
1993 or (realistically) 1994 or 1995. As in many areas of its analysis, the National Assessment will 
need to ascertain what states plan to do. This will be as important as determining what states have 
done by the time the National Assessment reports to Congress in mid- 1994. 

n. EFFECTS ON LOCAL EDUCATIONAL PRACTICES 

A second question which the National Assessment must address— indeed, the only question of 
any real interest, as I argue in the final section— is how the Perkins Act affects local practices in 
vocational education and, because the potential influence of the Act spills into the "academic** 
curriculum, in academic education as well. On the subject of classroom practices, four sources of 
baseline information can be cited: 

1. A survey of states and local districts in the prior National Assessment (Milsap et al., 
1989, reported in Muraskin, 1989, Ch. 1). This survey principally asked about the allocation of 
federal fiinds— including the use of various set-asides in the original Carl Perkins Act— and program 
improvement funds. 

2. Data on secondary transcripts, available for the high school class of 1972 from the 
National Longitudinal Study of the Class of 1972 (NLS72); the class of 1982 from the sophomore 
cohort of the High School and Beyond (HS&B) survey; the class of 1987 from a National Assessment 
of Educational Progress (NAEP) transcript study; and the class of 1990 from another NAEP transcript 
study. Transcripts for postsecondary students are available from the NLS72, for the high school class 
of 1972, and for the senior cohort in the HS&B data. 3 Such transcript studies are important because 
they reveal the frequency and coherency of coursetaking in vocational education. Prior examinations 
have revealed that many high school vocational programs are incoherent (e.g., Hoachlander and 
Choy, 1986) and high dropout rates from postsecondary vocational programs (e.g., Grubb, 1989a, 
1989b). However, the drawback to the currently available data sets is that none of them describe 
course taking patterns after the enactment of the Perkins legislation. Therefore, they can serve to 
indicate what practices have been, but they cannot indicate whether the requirement for "coherent 
sequences of courses** in Section 235 has had any influence. 

3. Research sponsored by NCRVE on the integration of academic and vocational education 
describes practices in effect in secondary schools as of 1990 (e.g., Grubb et al., 1991; Bodilly et al., 
forthcoming), based on visits to numerous schools around the country. An earlier survey by Losh, 
Border, and Bishop (1988) presents information from state directors of vocational education about 
high school integration efforts in their states. In addition, a telephone survey sponsored by NCRVE 
of about 300 community colleges and technical institutes will indicate what efforts to integrate 
academic and vocational education are now being undertaken at the postsecondary level (Grubb and 
Kraskouskas, forthcoming). These studies provide some guidance about the dominant approaches to 
integration. In addition, NCRVE research can also identify which states and localities have begun 



3 Postsecondary transcripts were also collected for the sophomore cohort of HS&B only for those 
students entering postsecondary education directly after high school, but this is a limited and essentially 
worthless sample. 
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steps toward integrating academic and vocational education, enabling the National Assessment studies 
to differentiate between schools that have already started reforms and those that will probably start in 
1991, when the new Perkins Act takes effect. 

4. A set of observations of vocational classes in samples of high schools and postsecondary 
institutions, undertaken by Weber et al. (1985). 

Given these sources of prior information, one obvious research method to generate evidence 
about changes is to resurvey local programs about their spending patterns and practices, roughly 
following the questionnaire used in the previous NAVE study. This survey should concentrate on the 
kinds of information that can be compared with prior results and can be readily determined by 
questionnaire, such as information about formal policies. 4 Such a questionnaire is also appropriate 
for asking about state policies— in both vocational and nonvocational areas— that have affected local 
practices. 

Further analysis of trends in patterns of vocational courses would be valuable, particularly at 
the secondary level. While such an analysis cannot reveal anything about the effects of the Perkins 
Act, it can indicate trends prior to the enactment of the Act. 

The crucial issue at the local level is how practices have changed in response to the Perkins 
Act, particularly in response to the requirement that programs "integrate academic and vocational 
education... through coherent sequences of courses so that students achieve both academic and 
vocational competencies" (Section 235). From experience examining schools that claim to have 
integrated academic and vocational education (Grubb et al., 1991; Bodilly et al., forthcoming; 
Adelman, 1989), it is clear that information about curriculum content and teaching methods cannot be 
reliably obtained through telephone questionnaires of written descriptions of programs. The reasons 
for this include the fact that administrators may engage in wishful thinking about their programs, and 
many changes are simply too complex and incomplete to be quickly understood. Therefore, the 
research undertaken about local practices should concentrate on site visits to a sample of secondary 
schools and postsecondary institutions. The site visits should determine how practices have changed, 
how integration of academic and vocational education is defined, the implementation problems 
encountered, and the school's or institution's plans for the future. (For greater detail about local site 
visits, see Stasz and Grubb, 1991.) The case studies are also appropriate for ascertaining what state 
policies have encouraged or hampered changes. 

In these local case studies, several obvious problems will arise. The first is that, while the 
possible conceptions of integration are relatively clear for secondary schools, the issues at the 
postsecondary level are not as clear. Research in progress by this author and Eileen Kraskouskas, 
sponsored by NCRVE, is currently surveying about 300 community colleges and technical institutes. 
The preliminary results indicate that the dominant form of integration consists of requiring vocational 
students to take general education classes. A few institutions are developing new curricula, such as 
vocational or technical mathematics and technical writing, and others are using Writing Across the 



4 In the area of vocational education, questionnaires often prove to be unreliable ways of collecting 
such ostensibly "hard" information as data on enrollments, enrollment trends and patterns, post-school 
employment, and the like, partly because the lack of data forces guesses of unknown validity and because 
the lack of standard definitions make comparability across institutions suspect. 
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Curriculum and Reading Across the Curriculum to change the content of both vocational and 
academic courses. About one-fourth of those surveyed seem to be on the road to more ambitious 
reform, and are, for example, designing new interdisciplinary courses. However, there has been 
much less experimentation at the postsecondary level than in secondary schools, and we expect that 
the dominant responses to the Perkins Act will involve either general education requirements or 
efforts to provide remedial education for students in vocational programs. One implication is that the 
National Assessment's research at the postsecondary level should be more exploratory, and probably 
restricted in scope, compared to the site visits at the secondary level. 

A second problem involves determining what has changed, particularly in schools where no 
direct information is available about practices prior to the 1990 Perkins Act. This is another case 
where site visits are clearly superior to questionnaire methods. In interviewing a series of teachers in 
a school over several days, and in observing classes, it is much more likely that researchers can get 
consistent answers about how practices have changed at the classroom level. Similarly, it will be 
crucial to determine the plans for subsequent change, since changes extending over several years will 
not be complete by the time the National Assessment study is due. 

In examining local institutions, it is crucial to distinguish between local policies and patterns 
of funding on the one hand, and changes in educational practice within classrooms on the other. 
Compliance-oriented administrators often focus on the former, as do federal regulators and auditors; 
but the interesting issue at the local level is how classroom content and interactions change. This is 
yet another reason for site visits rather than questionnaire methods. The administrators who typically 
respond to questionnaires are likely to confound changes in funding with changes in practice, and only 
classroom observations and teacher interviews can disentangle the two. 

Finally, there is one issue about local practices and requirements of the Perkins Act that is 
nearly intractable. The legislative language authorizing the National Assessment specifies that the 
assessment shall include "evaluations of... academic and employment outcomes of vocational 
education, including analyses of.. .the extent and success of integration of academic and vocational 
curricula" (Section 403). This suggests that outcome evaluations of integration efforts are necessary. 
However, evaluating the consequences of integration poses several technical problems. 

Programs being implemented under the banner of integration vary widely in their purposes, 
scope, and stage of development. This makes it difficult to compare outcomes across programs, or to 
devise appropriate control or comparison groups. In addition, appropriate measures of outcomes are 
unclear, in part because schools have turned to integration for varied reasons, and in part because 
well-known tests of cognitive abilities are clearly inappropriate to curricula that integrate vocational 
with academic education. Finally, the timing problem is impossible to resolve. Very few programs 
will be far enough along to expect any substantial outcomes by the time Congress reconsiders the Carl 
Perkins Act. For all these reasons, formal evaluations of the effectiveness of integration efforts are 
not warranted, though it may still be possible to carry out evaluations in specific and restricted cases 
(outlined in Stasz and Grubb, 1991). 
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III. EFFECTS OF ARTICULATION BETWEEN SECONDARY AND 
POSTSECONDARY PROGRAMS 



Interest in various forms of articulation between secondary and postsecondary 
programs— labeled variously as 2+2 programs, tech-prep, or simply articulation agreements— has 
been high throughout the last decade. As a result, there are several surveys of such programs in the 
late 1980s, including one by Long et al. (1986) and another by the National Commission on 
Occupational Education (1989) based on a limited sample of community colleges. In addition, 
ongoing research sponsored by NCRVE (Dornsife, forthcoming) will survey a large number of 2+2 
and tech-prep programs to understand variations in the programs* accomplishments. 

Under the Perkins Act, states (rather than the federal government) will allocate federal funds 
for tech-prep through an RFP mechanism. They therefore have the ability to establish their own 
policies about what constitutes tech-prep, how funds are to be spent, and whether other state and 
federal funds (including other Perkins allocations) are also used to support tech-prep. Therefore, an 
obvious research task is to survey states to determine their policies toward tech-prep and how Perkins 
funds earmarked for this innovation have been allocated. Such questions could be included in the 
survey of state policies mentioned in Section I above. 

As in the case of curriculum integration, it is necessary to examine separately how local 
programs change in response to federal tech-prep funds. The practices included in various tech-prep 
programs vary enormously, and, unlike the case of curriculum integration, site visits do not 
necessarily help clarify all of them. Therefore, a questionnaire addressed to a subset of the tech-prep 
programs funded through the Perkins Act, with different segments addressed to the secondary and the 
postsecondary institutions involved, seems the most appropriate way to determine what practices are 
being federally supported. 

From existing research on tech-prep programs (e.g., Long et al., 1986; Dornsife, 
forthcoming) it is clear that, just as there are many approaches to integrating academic and vocational 
education, there are also many different practices local institutions label "tech-prep." One of the most 
difficult issues will be to distinguish among the various approaches. In the simplest of these, 
postsecondary institutions simply provide information about their offerings to secondary schools in 
their region, hoping to attract more students. The most common practice is articulation (i.e., 
providing common course numbering systems for secondary and postsecondary courses taken in high 
school). This enables students to avoid repeating courses and enables them to complete postsecondary 
credentials faster, but otherwise leaves both secondary and postsecondary curricula unchanged. 

A substantial difference occurs when secondary and postsecondary institutions collaborate to 
change the curriculum of one or both institutions. A high school might change its vocational 
curricula; for example, by adding more science and math, incorporating applied academics courses, 
or creating a cluster approach to better prepare students for a particular postsecondary program. 
Secondary and postsecondary institutions might create new programs with a logical progression of 
courses during high school and on through a community college or technical institute. (In some 
cases, tech-prep programs make changes in the high school curriculum that amount to integrating 
academic courses into vocational programs, effectively combining these two reforms.) When 
substantial curriculum modifications take place, tech-prep programs can enhance skill levels rather 
than simply accelerate the pace at which students complete programs (as with articulation 
agreements). In addition, this approach to tech-prep can be interpreted as a way of reforming high 
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school vocational programs by making them broader, more rigorous, and better connected to 
postsecondary education, as well as to employment opportunities. 

Finally, in order to help students make better choices about their occupational futures and 
schooling plans, some tech-prep programs have improved guidance and counseling. 

Tech-prep programs that have become fully institutionalized usually incorporate several 
changes; that is, they are likely to have a sophisticated information and marketing campaign, modified 
curricula, and strengthened guidance and counseling at both the secondary and postsecondary levels. 
They also have established close working relations between secondary and postsecondary teachers and 
administrators. Such an approach is a far cry from a program that merely provides information to 
high school students or uses articulation of courses. 

The first responsibilit^lof a survey of local tech-prep programs is to distinguish the approach 
to tech-prep a particular program has taken. Of particular interest is the question of whether high 
school practices have changed in order to increase the competencies of high school vocational 
students. As with curriculum integration, there is a dynamic aspect to the question of what changes 
tech-prep programs have introduced. Because these reforms take several years, it is important to ask 
about planned changes in addition to those that have already taken place. 

Finally, it would be valuable for the National Assessment's examination of tech-prep 
programs to be asking some harder questions about the programs' effects. Since most tech-prep 
programs are, or recently have been, in the stage of development, they have often been surrounded by 
the boosterism that is an inevitable (and probably desirable) component of innovation. But to date, 
there is almost no information about some simple dimensions of tech-prep programs, including the 
numbers of high school students enrolled, the numbers that make the transition to the postsecondary 
component, and the numbers that complete associate degrees. Information about the subsequent labor 
market experience of these students is completely missing. What little evidence there is suggests that 
rates of progress through tech-prep programs are quite low, 3 raising the question of how these forms 
of articulation between secondary and postsecondary institutions can be made more effective. While 
it will be difficult for the National Assessment to develop very sophisticated outcome evaluations of 
tech-prep programs, it should be possible for surveys of local programs to ask for some measures of 
performance, and for the National Assessment to explore more systematic evaluations of those few 
tech-prep programs that have been in operation for some time. 

IV. EFFECTS OF VOCATIONAL EDUCATION ON LABOR MARKET OUTCOMES 

In a way, vocational education appears to be one of the few educational programs that should 
be simple to evaluate. Since one goal is almost by definition to prepare students for vocations, 
examining the labor market outcomes of vocational students (compared to similar students in 
nonvocational programs) should provide crucial evidence about the effectiveness of vocational 



5 Some of the only numbers on students* progress— from a tech-prep program at the Community 
College of Rhode Island— suggest that perhaps one quarter of seniors in a tech-prep program continued 
to the community college (Dornsife, forthcoming). However, these are rough estimates since the 
longitudinal data necessary to follow individuals through the different stages of tech-prep has not (to my 
knowledge) ever been collected. 
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programs. (Of course, as programs at the high school level shift to broader conceptions of vocational 
education, the validity of measuring success by employment alone becomes suspect.) Therefore, a 
large literature has developed examining the effects of vocational education on employment, 
unemployment, wages, and earnings. This literature is currently being reviewed by Charles Benson 
(forthcoming) and probably does not need yet another review. The only question is whether the 
National Assessment can sponsor additional analyses to address questions that have not yet been 
clearly addressed. 

The previous investigations of labor market effects have been limited by the data sets 
available. Each of these (e.g., the National Longitudinal Survey of the Class of 1972, High School 
and Beyond, and the National Longitudinal Survey of Youth) has been analyzed for the most obvious 
effects of vocational education. 6 While each data set has some substantial drawbacks, it is 
inconceivable for the National Assessment to invest in new data collection that could remedy these 
shortcomings (with one possible exception mentioned below). However, there are several issues 
which have been addressed in relatively few studies, which, in my view, the National Assessment 
could pursue to some advantage: 

1. Two studies, one at the secondary level (Rumberger and Daymont, 1984) and one at the 
postsecondary level (Grubb, 1991), indicate that economic returns to vocational education are much 
higher for those who find related employment and zero for those in unrelated employment (as one 
would expect if vocational education prepares individuals for a narrow range of occupations). 
However, the issues of how to measure "relatedness" and of how different conceptions of 
"relatedness" affect estimates of the proportion of vocational students finding related employment and 
the returns to vocational education have not been adequately addressed. This issue is especially 
important as states begin to develop performance measures, where one obvious measure of 
performance (already used by Florida to close programs) is the proportion of completers who find 
related employment. The National Assessment could, therefore, invest in research using currently 
available data sets to examine the effects of alternative conceptions of "relatedness/ 

2. To date, there has been little research on the dynamic effects of vocational education; that 
is, its effects over time as individuals move among jobs. One set of results (Meyer, 1981) indicates 
that the effects of high school vocational programs decay over time. Other results at the 
postsecondary level (Grubb, 1991) indicate that postsecondary certificates and associate degrees 
generally increase earnings, but do so by increasing access to those positions where individuals 
accumulate more experience and are more likely to receive on-the-job training. This implies that the 
benefits of postsecondary programs materialize only over time. The National Assessment could 
sponsor research using available data sets to analyze dynamic patterns more carefully. 



*One data set has not yet been examined, to my knowledge. The Survey of Income and Program 
Participation (SIPP) may be useful for examining the labor effects of postsecondary vocational education. 
The SIPP data asked individuals about their receipt of postsecondary vocational certificates and associate 
degrees, and also asked individuals whether they used their education on their current jobs. While the 
control variables in SIPP are relatively meager, these data could still shed some light on the effects of 
postsecondary programs and on the effects of education related and unrelated to current employment. 
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Analysis of dynamic patterns obviously requires longitudinal data, but relatively little exist 
that are appropriate to examining the effects of education on labor market performance. 7 One 
possible way to gain additional information about dynamic patterns would be for the National 
Assessment to support a limited survey of individuals, asking them retrospectively about their 
employment and education histories, their reasons for transitions, and the relatedness of their 
education and employment. This would be the only way to develop information on the current labor 
market experiences of individuals, and the only way within a short period of time to develop new 
information about dynamic patterns in labor markets. 

3. Most analyses of vocational education have concentrated on the effects of secondary 
programs; analyses of postsecondary programs have been much less common. This imbalance is 
understandable in the context of the 1960s, when secondary programs dominated. However, as 
enrollments in high school programs have dwindled, as secondary programs have turned to broader 
conceptions of vocational education less concerned with specific skill preparation, and as 
postsecondary vocational programs have expanded, it is clear that more research on postsecondary 
programs needs to take place. The National Assessment should, therefore, investigate further 
analyses of postsecondary vocational education, perhaps using data sets that have not been examined 
(see footnote on SIPP data.) 

There is one additional source of information now being developed that merits farther 
analysis. Unemployment Insurance (UI) data provides several potential advantages for analysis: 
1) It requires no expense for collection and does not suffer from the nonresponse problems associated 
with conventional surveys; 2) it is longitudinal; and 3) it is an accurate record of wages and earnings 
(though not other forms of income). Several states have examined the possibility of using UI data to 
examine educational outcomes (Baj, Trott, and Stevenson, 1990). Since these early efforts seem 
promising, the National Assessment could invest in further development of the methods necessary to 
analyze UI records to examine the effects of vocational programs, both to provide information about 
the labor market effects of vocational education and to provide guidance to states as they develop 
performance measures. 

V. GENERAL CONSIDERATIONS IN NATIONAL ASSESSMENf^ESEARCH 

In designing a program of research, some issues cut across individual pieces of research. It is 
important to keep these in mind during initial planning in order to reap the benefits of interactions 
among projects, since once research is designed and begun, it is usually too late to incorporate 
questions and issues that may affect other projects. 

One obvious consideration is development of corroborating information across separate 
studies. For example, information about state policies can be gathered not only through 
questionnaires to state officials but also in questionnaires to local programs and in local site visits. 
Information about changes in practice can emerge from examining changes in financing patterns, 
formal policy, and classroom practices. Information about efforts to integrate academic and 



Probably the best longitudinal data for these purposes is the NLS72 data, with 14 years of 
information on employment and postsecondary education for a sub-sample of individuals. The HS&B 
has only 6 years of data and the NLS data have relatively crude measures of education. None of these 
describe the experiences of students recently in school. 
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vocational education may provide evidence about tech-prep programs, and vice versa, since the most 
thorough approaches to integration usually establish substantial articulation with postsecondary 
institutions, and the most sophisticated tech-prep programs shape high school curricula in ways that 
look like forms of integration. The National Assessment should plan its research to maximize the 
possibility of corroboration across specific projects. 

One research tactic that has proved valuable within NCRVE is use of the local community as 
the unit of analysis; that is, investigating all related programs within a community rather than 
examining a variety of programs in different communities. One advantage of choosing communities 
is that programs within a community face similar labor markets, fiscal conditions, political 
configurations, and demographic patterns, reducing the variation in possible influences. In addition, 
the institutions within communities almost always interact with each other, even in communities where 
coordination seems relatively absent,' and information about these interactions helps clarify the 
transitions among institutions, the patterns of cooperation versus competition, and the processes of 
reform. One implication of this approach is that the National Assessment could examine both 
secondary and postsecondary institutions within a sample of communities, rather than choosing 
secondary and postsecondary programs from different communities, and could use these institutions to 
address a variety of congressionally mandated concerns. 

A third general consideration is the timing problem. Initial implementation of the Perkins Act 
will take place in 1991-92, but may, in practice, be delayed because the federal regulations 
accompanying the legislation had not been completed as of May 1991. With a report due to Congress 
in mid-1994, the National Assessment will be able to collect information about responses to the 
Perkins Act only through the beginning of the 1993-94 school year, the third year under the new Act. 
Given that many reforms instituted by the Act require difficult and subtle changes—particularly the 
changes in teaching implied by the integration of academic and vocational education, the novel 
linkages among institutions in tech-prep programs, and the transition to outcome-based policy implied 
in performance measures— it is unreasonable to expect that much will have changed by late 1993. It 
is crucial, therefore, to look for evidence of change-in-progress as much as reforms already in place 
and to avoid creating unreasonable expectations about what federal legislation can accomplish in a 
short period of time. 

A final consideration affecting all of the National Assessment's research is quite different 
from the others. The initial implementation of the Perkins Act coincides with a national fiscal "crisis" 
precipitated by the 1990-91 recession, in which the majority of states have discovered deficits and are 
seeking to cut their spending. Because of the magnitude of educational spending, the cuts will be at 
the expense of schools and colleges. Also, an alarming number of local districts are at the verge of 
bankruptcy. Indeed, the fiscal problems appear to be most severe in central cities, which are 
precisely die districts where Perkins funds will be directed under the new funding formulas of the 
Perkins Act. Reforms are difficult enough in periods of tranquility and prosperity, but in periods of 
fiscal pressure they may be nearly impossible because of more basic concerns about survival, among 
both administrators and teachers. In addition, schools desperate for revenues may welcome federal 



•For an analysis of the relations among various education and job training programs in specific 
communities, see Grubb and McDonnell, 1991 ; for a similar analysis of the remedial education programs 
linked to vocational education and job training, see Grubb et al., forthcoming. 
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resources, not for the reforms they can finance, but for the funding they can provide to compensate 
for lost state revenues. 

One possibility National Assessment research must examine is that reform efforts will be 
swamped by the fiscal crises some states and localities experience. It would be inappropriate to 
blame federal legislation for failure to reform vocational education if the next few years are a period 
of fiscal crisis and fights for survival. Should this prove to be the case, it will be necessary to 
confront the limitations of existing federal policy in K-14 education (now concentrated on marginal 
improvements to existing programs, particularly on behalf of low-income or "special needs" students) 
when the state and local funding base for education has become so vulnerable to fiscal cycles. 

VI. WHAT DIRECTION FOR FEDERAL POLICY? OR 
WHERE DO "WE" WANT TO GO? 

In the strictest sense, the National Assessment can comply with its mandate in the Perkins Act 
by documenting what happens in response to these changes in federal legislation. In a larger sense, 
however, the research the National Assessment undertakes— indeed, all research, whether policy 
oriented or not— should be driven by some sense of the basic problems at hand and the direction 
Congress wants to take with vocational education legislation. But therein lies the problem: As long 
as federal policy is the creation of consensus among groups with different interests and different 
perceptions of the problems, there is no simple interpretation of the direction Congress might have 
intended to take. However, the past 30 years of federal policy in this area provide some basis for 
interpretation. 

As noted in the introduction, the twin goals of federal vocational policy since 1963 have been 
directing federal funding to students with greater needs; that is, low-income and minority students, the 
handicapped, women seeking nontraditional occupations, and others now described as "special needs" 
students, and program improvement. Each goal has been subject to different ambiguities. In the case 
of program improvement, prior to 1990, Congress had never defined improvement. Therefore, local 
institutions had been free to define it for themselves and, in practice, had used federal funds to buy 
equipment and start new programs. These are forms of improvement that might be termed relatively 
routine; that is, they ought to happen with state and local ftinds as conditions change. Congress 
evidently felt it was not getting much for its money. It therefore defined program improvement in the 
1990 Act as the integration of academic and vocational education and other broader conceptions of 
vocational education. But rather than creating clarity, this may only continue to make program 
improvement murky. "Integration" is not defined in the legislation, it is unlikely to be further defined 
in regulations, and there is no indication that Congress had any idea what integration entails when it 
wrote the legislation. Indeed, while Congress does seem to have substantial change in mind, the 
overwhelming bulk of the Act's provisions cover administrative requirements, bureaucratic process, 
and the other minutiae of operating programs. It is all too easy to approach the Perkins Act as an 
administrative problem rather than a spur to change the direction of vocational education. 

There has been a different but equally consistent dilemma about federal requirements for 
special needs groups. It has never been clear why federal resources should be concentrated on these 
students in the absence of information that vocational education is beneficial to them. Indeed, since 
the preponderance of evidence about the labor market effects of vocational education has been 
negative, or at best neutral, one could read the persistent effort to focus resources on "special-needs" 
students as a nefarious attempt to get more lower-class and minority students into a "dumping 
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ground" or "warehouse," away from the rtiiddle-class kids in the academic track and away from the 
college-bound curriculum that might do them the most good. Similarly, in the Perkins Act, the 
persistent requirements about special-needs students and the ftjnding mechanisms that allocate 
resources to those districts and institutions with the highest concentrations of special-needs students 
may be premature. In the absence of evidence that federal funds are being used to improve 
vocational programs and make them broader and better integrated with academic competencies, it 
makes no sense to direct special needs students into them. The appropriate goal, assuming that 
Congress wants its vocational education policy to concentrate on low-income and other special-needs 
students (as almost all of federal education policy does), is that special-needs students should have 
greater access to the broader and more integrated forms of vocational education promoted by the Act. 

From these considerations, I conclude that the one issue of central importance, to which all 
others should be subordinated, is whether federal policy can change local educational practice in 
vocational education to move toward programs integrated with academic education, "coherent 
sequences of courses," content relevant to "all aspects of the industry," and generally broader goals. 
Until we can answer this question, other goals and issues, including the focus on special needs 
students, the desire not to supplant local funds, the governance provisions, and the many bureaucratic 
requirements that dominate the written form of the Perkins Act, are relatively unimportant. 

This central issue holds at least four implications for the National Assessment's research: 

1. All pieces of research should be connected to local changes. The effects of the Perkins 
Act on state policies, funding patterns, or access by special needs students (i.e., investigations 
of whether the use of funds meets the letter of the law) are irrelevant unless they can be 
related to changes in local practices. This means that each research task should not only 
examine what changes take place, but also should clarify whether, and how, such changes 
have affected local practices. 

2. School-level observations should be the core of the National Assessment's research 
because only with this method will it be possible to describe changing practices in vocational 
and academic education in response to the Act. 

3. If the purpose of federal policy is improvement in local educational practice, what forms 
might federal policy take other than the form it has always taken in vocational education (i.e., 
providing marginal amounts of money to a large number of districts and postsecondary 
institutions with (often vague) requirements attached)? Are there other forms that might be 
more effective (e.g., focusing on teacher preparation, funding more experiments or rigorously 
evaluated demonstration projects), 9 or supporting a small number of innovative schools? The 
National Assessment should keep in mind from the outset the need to examine alternative 
forms of federal policy once its research is completed. 

4. One research task necessary to address the previous question would examine what federal 
(or state) policies have been most effective in promoting changes in educational practice. 



*One curiosity is that there has always been considerably more funding for rigorous evaluations of 
experiments and demonstration projects in the areas of job training and welfare-related training programs 
than there has been in vocational education. 
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However, conventionally, this issue is analyzed by looking at a specific program to determine 
its influence. Often results show that its influence has been marginal (e.g., McLaughlin, 
1989). However, the question could also be answered by reversing the research procedure. 
Examples of changed practice can be analyzed to determine the causes of the change. There 
are several clear examples of change which could be examined from this perspective, such as 
the growing popularity of cooperative learning and the adoption of mastery learning and 
competency-based instruction (including competency-based instruction in vocational 
education). The results might be more instructive for fbrmulative federal policy than once 
again documenting the fact that a specific federal program has failed to make much difference 
because of implementation problems. 

The issue of whether and how federal policy can improve classroom practices extends beyond 
vocational education. The National Assessment might make common cause with the "academic" side 
of the Department of Education in sponsoring research along these lines. The results would be 
valuable to everyone, because without better ideas about how content and teaching within classrooms 
can be changed, we are likely to find that federal funding for program improvement in vocational 
education once again has failed to influence practice. 
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Effects of the 1990 Perkins Act on 
State Administration and Policy 



Phyllis A. Herriage* 



INTRODUCTION 

This paper discusses the effects of the 1990 Perkins Act on state administration and policies 
from the perspective of a state practitioner. Most of the content is based on observations and 
experience rather than research. Included are discussions about the role of the state in both 
administration and leadership and how some of the changes in the 1990 Act affect those roles. This 
discussion should assist researchers in identifying issues to be considered in the development of the 
study. 

The changes in the Perkins Act have major implications for state administration of vocational 
education, both in policy and practice. Some implications are obvious because of the fiscal impact 
created by the reduction in funding available for state administration and state discretionary activities. 
Other implications will vary according to the governance and delivery of vocational education in each 
state. 

In approaching this subject, it is important to keep several factors in mind. The variance in 
state governance structures, the variety of delivery systems operating among the states, the size of the 
state, and the state resources available make an analysts of impact and effect very complex. For 
example, one midwestern state administers vocational education through a separate agency with a 
large staff and adequate funding. The fiscal effects of the Perkins Act on state administration will be 
minimal in that state, but some of the functions performed by the state staff will change. In another 
midwestern state, state staff salaries are totally dependent upon federal funding, with the state 
matching funds, and Title II-B funding is used for technical assistance from the state office. In this 
state, the changes in the Perkins Act will dramatically affect the number of staff and resources 
available and the functions performed. 

As this study is developed, it will also be important to distinguish between the effects of the 
Perkins Act on state administration and the effectiveness of state administration in implementing the 
Act. Both areas are important, and information should be available about them. 

EFFECTIVENESS OF STATE ADMINISTRATION IN IMPLEMENTING PERKINS 

To study the effectiveness of state administration in implementing the Perkins Act, 
information can be gathered directly from states and from work completed recently by at least two 



♦The author is former Director of the Bureau of Technical and Vocational Education, Iowa 
Department of Education, and is currently Associate Director of the National Center for Research in 
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other agencies. During 1987-88, RAND/NCRVE (National Center for Research in Vocational 
Education) surveyed secondary and posts econdary policies in all SO states. Staff members of the 
Office of Vocational and Adult Education, U.S. Department of Education, conducted site visits to 
each state during 1989-91 to gather information about the way states define accountability and manage 
their accountability systems, and the impact on program quality. This review focuses on nine specific 
questions to be answered by the state and at the local level, including perceptions from selected 
employers of vocational program graduates. These questions are: 

1. How does the state define accountability? 
Program quality? 

Program impact? 
Program improvement? 

2. What is the state's process for determining program accountability? 
What sort of indicators are used to determine 

- Program quality? 

- Program impact? 

- Program improvement? 

3. How often is the accountability process used? How often and in what manner is the 
accountability process re-evaluated for its continued effectiveness? 

4. What is the role of state staff in using the accountability process? 

5. In what way, if at all, is the State Council for Vocational Education involved in the 
accountability process? 

6. To what extent, if at all, are outside public agencies, employers, private concerns, and 
individuals involved in the accountability process? 

7. How well does the accountability system measure program impact (i.e., the 
effectiveness of programs in meeting their stated objectives)? 

8. How does the system provide for followup on programs needing improvement to 
determine if, in fact, improvement in quality and impact has taken place? 

9. How does program improvement relate to program accountability? 

The answers to these questions and the data available from the RAND/NCRVE study should 
provide a baseline from which to develop a comparison for practices and impact of the new Perkins 
Act on state administration. 

EFFECTS OF PERKINS ON STATE ADMINISTRATION 

State Role 

During the reauthorization process, the role of the state received a great deal of discussion 
and study* The most obvious role of the state is to manage implementation of the legislation and 
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carry out regulatory responsibilities. But beyond those functions, states conduct activities to assist 
with local program improvement and leadership development through a variety of methods. Although 
the August 2, 1990, Conference Report contains very little discussion about the state role, the Senate 
Bill (S. 1109) Report provides some clarity about the role of states and expectations for states in 
implementing the Perkins Act: 

\ . . the Committee recognizes that there is a significant role for the states to play in carrying 
out programs in two important areas: curriculum development and teacher training/ (p. 15) 

"The Committee recognizes that effective administration of vocational education programs can 
only be accomplished with fiscal support, but it is the Committee's clear intent that the 
primary use of the federal dollar should be that of direct services. 

"Because S. 1 109 changes the Perkins Act and places new responsibilities at the local level, 
the role of the state is similarly redesigned [emphasis added]. By driving money to the local 
level, S. 1 109 frees up the state to monitor and evaluate programs, set goals and priorities, 
emphasize program improvement and excellence, and provide the kind of state leadership that 
will encourage programs to administer the economic needs of the state. 

. . in complying with S. 1 109, the states are to spend the majority of their allocated funds 
on programs that benefit local providers." (p. 16) 

The 1984 Perkins Act enabled Title II-B funding to be used for technical assistance. Much of 
this technical assistance was conducted through the state office, either directly or through contract 
with an educational institution, typically a university. A number of individuals and groups 
representing local educational interests expressed the concern that to a greater extent than the law 
intended, program improvement dollars meant for local use were being used by the states for 
statewide activities. As there authorization process unfolded, it became clear that steps would be 
taken to ensure that a greater portion of federal funds would be directed into local programs for 
improvement at the discretion of the local administration. As a result, states have eliminated some of 
the statewide activities which include a variety of services provided for the local level. The National 
Assessment could identify which statewide activities and services were eliminated and evaluate the 
effect on the quality of programs at the local level. 

In addition to directing funding to the local level via formulas, the new Act reduces the 
remainder of the funding available to states for statewide activities and limits the use of funds to items 
prescribed in the law: [Sec. 201(b) and (c)]. 

H (b) REQUIRED USES OF FUNDS. -The programs and activities described in subsection 
(a) shall include- 

(1) professional development activities for vocational teachers and academic 
teachers working with vocational education students, including corrections 
educators and counselors, and educators and counselors in community-based 
organizations, including inservice and preservice training of teachers in state- 
of-the-art programs and techniques, including integration of vocational and 
academic curricula, with particular emphasis on inservice and preservice 
training of minority teachers; 

(2) development, dissemination, and field testing of curricula, especially- 
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(A) curricula that integrate vocational and academic methodologies; and 

(B) curricula that provide a coherent sequence of courses through which 
academic and occupational skills may be measured; and 

(3) assessment of programs conducted with assistance under this Act, including 
the development of- 

(A) performance standards and measures for such programs; and 

(B) program improvement and accountability with respect to such 
programs. 

"(c) AUTHORIZED ACnVITIES.--The programs and activities described in subsection 
(a) may include- 

(1) the promotion of partnerships among business, education (including 
educational agencies), industry, labor, community-based organizations, or 
governmental agencies; 

(2) the support for tech-prep education as described in section 344; 

(3) the support of vocational student organizations, especially with respect to 
efforts to increase minority participation in such organizations; 

(4) leadership and instructional programs in technology education; and 

(5) data collection/ 

The reduction in funds available for state-administered activities and state administration, 
coupled with the changes mandated for program structure, content, and emphasis, will cause states to 
examine and probably redefine the role of the state staff. This will probably result in new job 
descriptions and responsibilities for some staff. If the changes in the Perkins Act are to be 
implemented effectively at the program level, major statewide staff development and curriculum 
redesigning must occur in such areas as integration of academics and vocational education, 
appropriate services for special populations, competency-based vocational education, and teaching "all 
aspects of the industry/ To what degree is this occurring? What new trends have emerged? Have 
functions of the state staff shifted from specialization to generalization? 

The underlying questions in the 1990 Perkins Act are: What is the expectation for state 
administration? Is it only to facilitate funding flow-through? Is a leadership role expected? It is 
important for the state role to be as clearly defined as possible as a part of the state plan. Assessing 
impact and effectiveness of states in the delivery of vocational education will be difficult to measure 
without a clearly defined role. 

As states have begun the process of developing state plans, the fiscal and administrative 
impact of the changes in the Perkins Act has been realized. The following situations are occurring: 

• The responsibilities of many staff members are being redefined. This is a result of 
the legislation's changes in the processes carried out by the states. For example, the 
vocational evaluations which traditionally are were conducted by the state staff are 
now conducted at the local level. The state's role will be providing technical 
assistance rather than administering the evaluation process. 

• The legislation requires employment of certain staff during a time when state budgets 
and federal funds for state administration are reduced. These mandated positions may 
mean that some functions are eliminated or other staff are reduced. One state has 
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already indicated that it may decline the Title III-B funds for comumer-homemaking 
because it cannot meet the staff requirement. 

• Many states are experiencing budget problems and are unlikely to be able to assist 
state and local programs by replacing Perkins funds redirected via the formula. States 
that previously relied heavily on the use of Title II-B funds for the cost of staff to 
conduct technical assistance may have difficulty finding another source to replace 
those dollars. 

• The size of the state staff and amount of resources available varies tremendously all 
over the nation. Yet the requirements and functions which must be carried out to 
meet die mandates in the legislation and to provide leadership remain the same for all 
states. For example, it takes exactly the same amount of state staff time and effort to 
develop a state plan whether the state staff numbers 400, 40, or 4. Therefore, the 
quality and degree to which new mandates are implemented may be affected by the 
state's size. 

• Some states are reducing the number of staff because of the loss in funds available for 
state administration (reduced from 7 percent to 5 percent) and the limitations on funds 
available for other staff use. 

The National Assessment could identify the situations occurring in states which may affect 
state administration but are not a direct result of changes in the Perkins Act. This clarification could 
help distinguish the impact of the Act from other coincidental factors. 

The variety of approaches used by the states to develop state plans should also be considered 
in the study. For example, Section 1 13 of the Perkins Act requires that the state plan be developed 
on the basis of the results of a needs assessment, to be conducted on a very short timeline. 
Theoretically, the results of that needs assessment should identify priorities for the use of funds, 
technical assistance, staff development, and curriculum development, particularly for the new items 
contained in the Act. It should provide the benchmark against which progress can be measured. 
Some questions for research are: How did states approach the conduct of the needs assessment? 
To what extent does the state plan reflect the needs identified? How does the state respond to 
results that may indicate a need to review and make changes in the entire delivery system for 
vocational education? How does the state establish priorities for state-administered activities? 

The previous NAVE (National Assessment of Vocational Education) study devoted most of 
the discussion about state administration to the methods states used to distribute funding. Very little, 
if any, of the study focused on the actual impact of state administration on the quality of local 
programs. This appears to be the primary question for the new National Assessment. 

State Expectations and Local Control 

The law places increased emphasis on accountability and monitoring by the state. At the 
same time, it changes the role of the state in evaluating local programs. The new state-developed 
systems of performance standards and measures place emphasis on "outcomes," but the evaluation 
process is managed at the local level. Ideally, these changes should result in improved outcomes for 
students because the evaluation of the instructional process is managed closer to the classroom level. 
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This change, while not entirely new, certainly is a new way of "doing business" for many 
states. Most states have some type of program standards that measure program "inputs/ but fewer 
have fully developed performance standards which will measure the outcomes of vocational education 
instruction. It would be useful to assess the states* efforts in this area, using questions such as: 

• Will the shift from state control to local control of the evaluation process have an 
impact on results for students? 

• Will there be enough uniformity of results in a locally controlled system to draw 
conclusions about the effectiveness of vocational education programs? 

POLITICS IN EDUCATION 

The new requirements in the 1990 Perkins Act place vocational education in the position of 
pioneering some major components of educational restructuring, including outcomes-based education; 
state-established performance goals with processes managed locally; and, through requirements for 
integrating vocational and academic education, improvement of academic skills. 

Politics in education has the potential to make implementation of these changes very 
challenging. Because vocational education has held second-class status within the educational system 
for many years, it may be difficult for it to gain acceptance as a legitimate vehicle for teaching 
academic skills. To be successful, strong linkages must be immediately established between 
vocational education and the traditional academic education community. One small segment of 
education alone cannot effectively make the required changes such as integrating academics and 
vocational education. Turf protection and technical obstacles such as teacher licensure will need to be 
overcome. Many of these changes must start in the state office. In terms of the staff and credentials, 
state education agencies are often organized like a local school district. Therefore, for vocational 
education to take responsibility for components in educational restructuring, the vocational education 
unit in state government will need to have strong linkage with the rest of the education staff. The 
assessment might ask: What are the political and technical obstacles states identify and remove 
to accomplish the instructional mandates in the Perkins Act? (Some obvious examples would be 
acceptance by and cooperation from the academic community, and issues related to teacher 
licensure.) 

DELIVERY SYSTEM 

The previous NAVE study, Final Report, Volume //, discusses differences in the way states 
allocate funds under the Perkins Act. It found some variations and also drew several conclusions 
about state practices in allocating funds: 

• Nationally, almost 40 percent of funds were spent for postsecondary education. 

• Separate area vocational school districts appear to have received a disappropriate share 
of the federal funds that flowed to secondary education. 

• On a per-pupil basis, area vocational school districts and postsecondary institutions 
received much larger grants than school districts. 
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• If only school districts are considered, there is some evidence of targeting funds to 
districts with higher proportions of minority populations. If Pell Grants are used is a 
measure of need, there is no evidence of comparable targeting at the postsecondary 
level. 

• In at least half of the states visited by the NAVE study team, the state office 
administering the Act was able to limit the grant eligibility of institutions not under its 
purview. For example, in a state where vocational education is administered by a 
postsecondary board, the participation by secondary districts may have been limited. 

• Most states had a predetermined portion of funds available for secondary and 
postsecondary education (or for particular institutions). 

• The unequal distribution of resources across sectors and institutions meant that 
intended beneficiaries in a given sector or set of institutions were considerably more 
likely to receive support in some states than in others. 

• Most school districts received awards that were too small to enable them to mount 
new initiatives of any size. 

All states have K-12 districts and postsecondary institutions that are involved in providing 
vocational education to some extent, but many also have a system of area vocational schools which 
have been the focal point in the vocational education delivery system. Changes in the methods for 
distributing funds will probably have the most impact on those area vocational schools. This shifting 
of funds among eligible recipients may be dramatic enough to cause a re-examination of the delivery 
system. 

Both the methods of allocating funds and the funding formulas will affect the overall delivery 
system for vocational education in most states. The high schools, which traditionally have not been 
recipients of large amounts of federal vocational education funds, will probably receive larger 
allocations than in the past. Districts may opt to keep the funding rather than allocate their share to 
an area vocational school. Some states where the area vocational schools have been the chosen 
delivery system will be the most affected by this change in allocation practices. 

A question for assessment is: As a result of new funding approaches, what changes, if 
any, will occur in the delivery of vocational education? 

SPECIAL POPULATIONS 

The 1990 Perkins Act places increased emphasis on serving special populations and requires 
that they be served in high-quality vocational education programs. This change affects state 
administration because states will need to provide strong leadership, conduct staff development, 
develop resource materials, and monitor the actual participation of special populations in vocational 
education. 

Because of the structure and funding requirements of the previous Perkins Act, the emphasis 
on special populations resulted in a focus on fiscal and accounting processes rather than on providing 
services. The majority of effort and time was spent on formulas, computing excess costs, and 

31 



ERJC 



36 



documenting numbers served. The best practices for helping special populations have access to and 
success in participating in vocational education received much less attention. 
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The NAVE study, Final Report, Volume /, states that "Little additional direction has been 
forthcoming from federal or state levels on the targeting of federal funds for supplementary services 
to disadvantaged and handicapped students, nor has technical assistance been provided on effective 
practices for serving these populations/ Some questions for assessment are; What staff 
development activities does the state conduct to assist local programs in providing appropriate 
accommodations for members of special populations? What resource materials has the state 
developed for use at the local level that identify effective strategies and methods for instructing 
members of special populations? How does the state provide leadership to assist local programs 
in serving members of special populations in high-quality vocational education programs? How 
does the state monitor the following elements for members of special populations in vocational 
education: access, actual participation, assessment of need, support services, and job 
placement/transition? 

States should be a critical link to successful vocational education at the local level. For the 
goals and mandates of the 1990 Perkins Act to be effectively implemented, states must play a strong 
role in assisting local programs with appropriate methods for serving special populations and 
implementing the other changes in the legislation. 
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A New Vision for Vocational Education: 
Assessing Implementation of the 1990 Perkins Act 



Lawrence Rosens tock* 



INTRODUCTION 

The 1990 Perkins Act offers vocational educators unique challenges and opportunities to 
reshape their practices in ways that will deepen the educational experiences of students and bring 
vocational education into the 21st century. The principal issues for the National Assessment of 
Vocational Education at the local level concern whether or not local players are seriously striving to 
implement key features of the Perkins Act (i.e., integration of vocational and academic education and 
"all aspects of the industry" instruction). "Integration" and "all aspects" represent a radical departure 
from past practice in vocational education. These new practices are not easily understood and may be 
difficult, complex, and time-consuming to implement. Oddly, most vocational practitioners claim that 
they are already doing these things. 

Assessing these new practices and measuring deg» of implementation will be difficult. 
This paper will examine the dimensions which signal that schools are integrating vocational and 
academic instruction. It is critical that the National Assessment develop instruments that distinguish 
practitioners who are actually integrating academic and vocational education from those who are using 
the same terminology to describe practices that are not true integration. For example, the assessment 
should be capable of distinguishing between poles, where one program's integration stops at students 
occasionally using math to measure raw materials and another program's academic and vocational 
teachers have engaged in sustained, long-term planning and students' entire vocational and academic 
curriculum has been fully integrated and structured around learning about and operating within an 
industry. Thus, it would be important to know if integration of academic and vocational instruction 
appears in a program's objectives but is not afforded teacher planning time and does not appear in 
course materials. 

INTEGRATION OF VOCATIONAL AND ACADEMIC LEARNING 

In the 1989 hearings on the reauthorization of the Perkins Act, virtually every commentator 
noted the need to bring together these two parts of our educational system. The Act clearly requires 
integration in its sections on state assessments, state plans, local plans, local evaluations, and 
elsewhere. The challenge for local implemented is to translate this rhetoric into day-to-day reality in 
schools. 

Section 239(c)(1)(B) requires that basic grant funds be used in programs that "integrate 
academic and vocational education in such programs through coherent sequences of courses so that 
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students achieve both academic and occupational competencies/ The breadth of the meaning of 
"integration" is emphasized in both the House and Senate Reports. For example, the Senate Report 
states: 

"mhe local recipients are to use Perkins funds to support programs which integrate academic 
and vocational training. Such integrative activities are intended to support a sequential course 
of study which uses an applied context to teach both broad and specific job training and which 
at the same time reinforces academic skills." (p. 20) 

"The Committee notes that such integration must be supported by a two-pronged approach: it 
must include the development of the curricula, and it must include the teacher training 
required to make effective use of that curricula." (p. 10) 

Similarly, the House Report emphasizes "academic curriculum. >. directly connected with real 
life applications," including "more advanced skills of thinking, problem-solving and comprehension" 
(p. 6). It notes that important elements for integrating academic and vocational coursework include: 
"curriculum development and professional development," ensuring that "both vocational and academic 
instructors [are] involved from the start in the integration of the curricula," and the need for strong 
leadership "to break down the significant barriers that often exist between vocational and academic 
programs." 

Under the Act, in assessing "integration of academic and vocational education," it would not 
be adequate to look at students* transcripts or schedules and determine that integration is taking place 
because math, English, social studies, science, and vocational courses are listed. Rather, the inquiry 
must determine the extent to which curricula are synthesized and the level of development and use of 
basic and advanced academic skills in the vocational setting. There is no single, prescribed way to do 
these things, but a comprehensive approach is involved, with courses aligned, sequenced, and 
mutually reinforcing (both across students' current schedules and over time), and with academic and 
vocational staff working together. The degree of integration in the program must be assessed, not 
whether the student takes separate, unintegrated programs which may or may not become integrated 
in the student's mind. 

One barrier to integration is the nearly total separation between academic and vocational 
education in today's schools. In the Perkins Act, Congress seeks to deter the tracking that relegates 
some students to "work with their hands," with minimal and diluted academic content in their 
courses, as schools fulfill their own prophecy of these students' limited potential. The subtext of 
vocational education has been its function of sorting by social class: Vocational education is for the 
"non-college bound," (i.e., working-class) student. Pursuing integration with academic education 
requires us to take a different view which sees vocational education as an alternative way to learn the 
same academic concepts and skills (including research, analytic, and philosophical thinking skills) that 
other students must learn. This is a new expression of an old truth. Vocational teachers have always 
seen students demonstrate competence, thinking ability, and creativity in a vocational setting, although 
some students are unable to demonstrate those skills in an academic setting or on a standardized test. 

Vocational students demonstrate mastery through exhibition. Academic educators are now 
pursuing similar approaches which allow students to show individually configured excellence. As 
academic and vocational education integrate, academic education must go even further in embracing 
vocational methodologies, while vocational education must include much greater academic content. 
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"ALL ASPECTS OF THE INDUSTRY" INSTRUCTION 

The new Perkins Act provides a platform upon which to integrate academic and vocational 
education— "all aspects of the industry" instruction. "AH aspects" reflects a movement away from 
occupationally specific, narrow, skill-based training. The term "industry" should be interpreted 
broadly. In the legislative history, Congress expresses strong concern about overly narrow skill 
training. Also, the list of factors in the Act (planning, management, finance, underlying principles of 
technology, technical and production skills, labor, community, and health and environmental issues as 
they pertain to an industry) indicates that using the term "industry" instead of "occupation" is 
intended to broaden, not narrow, the focus of the training. Thus, the term is not intended to be 
limited to the highly specific definitions in the Standard Industrial Classification. Rather, it should be 
read broadly to cover such examples as the automotive or transportation industry, the construction or 
housing industry, the telecommunications industry, etc. Within this broad framework, local schools 
can use considerable discretion in how to name or define the "industry." 

This broad view is important for ensuring that the purposes of the Act are met. Focusing on 
"all aspects of the industry" ensures that the areas of planning, management, etc., are not defined 
away by a view that says they are not needed for the particular job or occupation for which the 
student is training. Congress was very concerned about developing a broad range of occupational and 
academic skills and not locking students into overly narrow skill training for one predefined job slot. 

Neither integration with academics nor "all aspects of the industry" instruction can be 
accomplished without the other. They are two ways of looking at the same educational goals. 
Integration means merging the head and the hand. "All aspects" means essentially the same thing; 
that is, broadening our currently narrow, occupationally specific definitions to encompass the skills 
(often conceptual and critical) required to understand management, planning, history, and the social 
aspects of broad industries. 

ELEMENTS OF INTEGRATION 

To fulfill the Act's promise, the people closest to the students at the local program level 
should have the strongest sense of the Act's goals, what the local evaluation is looking for, and how 
to best document student learning through integration of vocational and academic instruction. In 
addressing the range of local curricular and programmatic issues regarding academic integration and 
"all aspects of the industry" instruction, the following elements should be considered: 

• teacher planning and coordination 

• curriculum emphasis 

• course materials 

• student and teacher scheduling 

• new teacher training and staff development 

• work-based learning 

• student assessment 
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Teacher Planning and Coordination 



Integration of vocational and academic education is no small feat. Faculty from die various 
disciplines must work together to align and sequence coursework and to make it mutually reinforcing. 
Local administrators must allow substantial planning time for this work. 

It is not enough merely to export a few vocational competencies into the academic setting, nor 
merely to import a few academic competencies into the vocational setting. These practices are self- 
reported by many practitioners as representing adequate integration within their schools. However, 
schools can take certain steps to indicate that they are doing more. To get at those steps, the 
assessment must ask: Do vocational and academic teachers see their mission as joined? Are they, 
collectively, working up their own lesson plans, including observations, experiments, readings, 
demonstrations, writings, and other things that fit within the flexible system? 

When teachers in various disciplines produce their own teaching materials and thus gain 
ownership over those materials, they find themselves less threatened by the "difficulty" of other 
teachers' subject matter. Teachers attuned to the methods and materials of the language arts and 
social studies find that the methods, language, and thought processes of the sciences can easily be 
conjoined with the materials they are using in their disciplines. Inversely, scientific materials and 
subjects (including vocational ones) can be enriched and enlivened by the application of a more 
humanistic, qualitative, and consciously subjective approach. 

Data sources available to assess planning and coordination: 

• teacher schedules 

• inservice calendars 

• local staff development policies 

• state staff and curriculum development offerings 

• visits to exemplary programs 

Curricular Emphasis 

We experience reality seamlessly, not in fragmented disciplines. We do not ordinarily 
consider events and processes in terms of their distinct historic, physical, mathematical, and aesthetic 
components. Teaching should try to approach our experience of reality. The assumption here is that 
the constant switching of gears by teachers and students throughout the day, from subject to 
disconnected subject, results in an onerous fragmentation of learning. Academic curricula, like 
vocational curricula, suffer from a narrow conceptualization. In seeking to unify the curriculum, the 
new Act envisions an integrated, broad view of industries and their relationship to the economic and 
social realities of communities. 

Multidisciplinary efforts cannot be generated abstractly. There must be an organizing theme 
with enough power and coherence to drive the entire curriculum. An "all aspects of the industry" 
focus provides a foundation for creating such themes. For example, the study of environmental 
management is enriched by science and vocational teachers working together, and the study of 
affordable housing is enhanced by the collaborative efforts of carpentry, architectural drafting, and 
social studies teachers. 
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Education must move away from a process that teaches skills as separate entities to one that 
uses analogies and connections between skills as a way to sharpen students 9 powers of observation and 
expand their experience. This may help prevent students from being "turned off" by one or another 
specific subject. To further enrich this integrated learning, teachers must be urged to use as many 
resources from the community as they can muster. Bringing the outside world into the classroom 
(and the classroom to die outside world) will help break down the artificial, counterproductive 
division between school and the N reaT world. 

In its experimental, hands-on component, vocational education has its greatest strength* 
Fundamental to its approach is an understanding that some learners need to work with their hands to 
inspire and open their minds. By integrating disciplines and learning styles, students experience a 
holistic world which becomes a laboratory for acquiring knowledge and skills, and for learning to 
pose and solve problems. 

Data sources available for assessing auricular emphasis: 

• course catalogues 

• course objectives and syllabi 

• texts and other course materials 

• instructional methods 

• student assessments 

• class assignments 

• visits to exemplary programs 

Course Materials 

Vocational personnel are inundated with trade journals, association mailings, and staff 
development activities which emphasize a narrow-skill, occupationally specific focus. New materials 
must be developed and teachers must be provided opportunities to shape them. The practice of 
covering material only sufficiently enough to prepare for the next course in sequence must change. 
Rigid instructional patterns in which the entire class studies the same thing at the same time also must 
change* Reliance on textbook dominance, emphasis on "coverage," and student memorization must 
be reduced. 

Although there is no one best way to organize instructional materials, these materials must 
encourage students to explore concepts as they relate to the students 9 lives. Vocational materials must 
allow students to be actively engaged in broadly examining phenomena related to industries. These 
materials must encourage students and teachers to be active social inquirers and participants in their 
own communities. For example, vocational programs can explore and perhaps assist in providing 
low-income housing development, childcare, or services for the elderly. Active problem posing and 
problem solving should be a priority. Use of primary source materials should take precedence over 
use of secondary materials such as textbooks. Materials should involve student writing across the 
entire curriculum, including the vocational subjects, as a tool for learning. 

In the work world, individuals rarely operate in isolation. Yet most vocational instructional 
material deemphasizes cooperative or group learning (e.g., competency-based vocational education 
focuses on duties and tasks performed by die individual learner). Materials should focus on 
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individual understanding within group processes- They should promote group exploration, discussion, 
questioning, and explaining. 

Data sources available to assess instructional material: 

• exemplars from model programs 

• samples of district curricula 

• samples of state-mandated curricula 

• student writing samples 

• evidence that teachers are developing their own new materials 

• visits to exemplary programs 

Student and Teacher Scheduling 

Most secondary schools are constricted by anachronistic scheduling. They are hampered by 
rigid requirements for numbers of "contact hours" in each of the traditional Carnegie unit courses. 
Additional requirements for shop time compound the problem. An integrated, multidisciplinary 
approach seeks depth of student understanding, rather than breadth of "coverage" of various subject 
matters. 

Vocational administrators usually present scheduling as the primary roadblock to change. The 
schedule— the status quo calcified— fragments academic and vocational learning into unrelated periods 
throughout the day and week. This raises the question of whether it is possible to integrate vocational 
and academic instruction without physically integrating vocational and academic instructors. 

Much has been written about the isolation of teachers, yet little has been done to end it. Most 
teachers know little about the practice of other teachers whom they have worked close to for yettfe. 
Team teaching helps end isolation and combat the notion of the individual as sole transmitter of 
information. It also allows for the type of multidisciplinary benefit described above. Back-to-back 
scheduling (e.g., having a vocational and a social studies class share two periods) allows teachers to 
work together and creates opportunities for common planning time. In schools where these 
techniques are tried, teachers often feel liberated from former constraints and enjoy the unanticipated 
pleasure of collaborating. 

Data sources available for assessing scheduling: 

• master/teacher/student schedules 

• teachers' room assignments 

• staff development activities 

• district graduation requirements 

• state vocational class hour requirements 

• visits to exemplary programs 

• credit mechanisms for multidisciplinary study 

New Teacher Training and Staff Development 

Even a cursory examination of most states 1 teacher-certification requirements reveals a 
continued emphasis on narrow-skill training for future vocational educators. Similarly, course 
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catalogues for teacher-training programs at the undergraduate and graduate levels do not depart from 
sustaining the status quo. What forms of guidance from state and officials will best encourage these 
changes? Certainly, at the state level, the connection between teacher-training courses and teacher- 
certification requirements is great, as courses are generated which serve certification requirements. 
The assessment must closely examine the extent of state officials* commitment to bringing about these 
changes. Vocational educators are a maturing work force; therefore, reshaping the expectations of 
new practitioners entering the field has great potential for change, even in the near term. 

It is also critically important that state and local staff-development opportunities for teachers 
and requests for proposals for training reflect the Act's new requirements. 

Data sources available for assessing new teacher training and staff development: 

• state vocational certification requirement 

• teachers' college curriculum content and emphasis 

• state and local staff development activities 

• state requests for proposals for staff development 

• local initiatives to train and retrain teachers 

Work-Based Learning 

Cooperative placements, work-study, and apprenticeship programs can do more for students 
when they are integrated with academic learning and when they embrace an "all aspects of the 
industry" approach. These programs can go beyond the work experience, integrating work-based 
learning with school-based learning. Rather than funneling students into a single occupational slot, an 
"all aspects of the industry" orientation would diversify students' work placements and rotate them 
through various sectors of an industry. Thus, a student in a hospital-based program would move 
through the nursing, finance, dietary, and other departments. 

Experiential placements fail when students are the slender thread between the school and work 
cultures. An integrated structure, planned by school and employer staff, must be in place before the 
student enters the program. For example, a program at the Polaroid Company joins students, 
teachers, and work supervisors in daily seminars where all participants write reflectively about their 
common experiences. 

Data sources available for assessing experiential placements: 

• cooperative agreements between schools and employers 

• interviews with students and employers 

• examination of policies and procedures of model programs 

Student Assessment 

As vocational education integrates with academics, a broade r range of students* abilities must 
be assessed. Unlike typical classroom procedures that rely heavily on linguistic and mathematical 
abilities, applied learning activities such as those that take place in vocational classrooms place greater 
emphasis on learning in context. 
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New methods for student assessment must emphasize the highly individualized ways in which 
people learn. The new Act's emphasis on integration commits itself to students' deep understanding 
of and experience in the core aspects of industries. Student assessments must reflect students' use of 
that knowledge as they explore and seek to solve, within the context of their own communities, the 
problems that face those industries. Such efforts go far beyond the bounds of traditional assessment 
measures. 

Thus, as integration takes place and multidisciplinary learning ensues, new measures of 
student learning must be developed. The portfolio, a collection of student work, is coming into 
increasing use. Such portfolios for vocational students should include elements of student learning 
which reflect the new, broad, industry-wide focus. 

Data sources available for assessing student progress: 

• student portfolios 

• graduation rates 

• dropout rates 

• rates of participation of special-populations students in high-quality programs 

• college placement rates and other outcome measures 

CONCLUSION 

This paper focuses on ways to recognize meaningful change in local vocational programs. 
The National Assessment must determine what these new practices look like, and whether there is 
meaningful movement toward such practices. In this regard, it is important to identify schools with 
exemplary programs and undertake qualitative research which closely examines the process of 
implementation in these schools and disseminates the findings to others. 

It is also critical that the National Assessment analyze state and federal guidance. Local 
implemented cannot succeed without strong support and encouragement at all levels. The assessment 
should examine how this necessary support will be forthcoming. How can the U.S. Department of 
Education and the states best offer sufficient guidance to local practitioners to redirect their programs 
as required by the new Perkins Act? Will the call be clear for a new, dramatically different practice? 
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The Effects of Federal Requirements Regarding 
Articulation Between Secondary and Postsecondary 
Vocational Education Programs 



Roy Peters, Ji\ and Richard C* Makin* 



INTRODUCTION 

The Carl D. Perkins Vocational and Applied Technology Education Act Amendments of 1990 
(P.L. 101-392) mandate that the Office of Educational Research and Improvement conduct applied 
research on aspects of vocational education specifically related to the Act. One study mandate is "the 
effect of federal requirements regarding articulation between secondary and postsecondary vocational 
education programs." 

Undoubtedly, several aspects and provisions of the Act can potentially affect 
secondary/postsecondary articulation. These include: 

1. Vocational education programs that offer coherent sequences of courses leading to job 
skills. 

2. The integration of academics and vocational education through coherent sequences of 
courses so that students achieve both academic and occupational competencies. 

3. Tech-prep programs as an allowable activity under uses of funds at the local level. 

4. Title III tech-prep education programs that include a common core of required 
proficiency in mathematics, science, communications, and technologies, designed to 
lead to an associate degree or certificate in a specific career field. 

5. The establishment of postsecondary vocational education as a separate program in the 
federal legislation. 

The purposes of this paper are to present four major areas related to a national assessment of 
the effect of federal requirements on secondary/postsecondary articulation. These areas include 1) 
background information, 2) the issues to be investigated, 3) die principal data sources and methods of 
inquiry that might be used to investigate the issues, and 4) the problems to anticipate when 
investigating the issues. 



♦Roy Peters, Jr. is State Director, and Richard C. Makin is Coordinator of Research at the Oklahoma 
Department of Vocational and Technical Education. The authors acknowledge and thank Charles O. 
Hopkins, Assistant State Director, and Sheryi Hale, Graduate Research Assistant, for their assistance in 
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BACKGROUND 



Hull and Parnell (1991) broadly define articulation as a process for linking two or more 
educational systems within a community to help students make a smooth transition from one level to 
another without experiencing delays, duplication of courses, or loss of credit. A decade earlier, Con 
and Hardy (1978) described articulation as a process, an attitude, and a goal: 

As a process, articulation is the coordination of policies and practices among sectors of the 
education system to produce a smooth flow of students from one sector to another. As an 
attitude, it is exemplified by the willingness of educators in all sectors to work together to 
transcend the individual and institutional self-interest that impedes the maximum development 
of the student. As a goal, it is the creation of an educational system without artificial 
divisions, so that the whole educational period becomes one unbroken flow, which varies in 
speed for each individual, and which eliminates loss of credit, delays, and unnecessary 
duplication of effort. 

In recent years, increased emphasis on articulation has been stimulated by state and federal 
mandates and a growing interest in educational reforms. In a review of articulation and transfer, 
Kintzer (1989) reported that 30 of the SO states currently had transfer guidelines or policies 
encompassing the major segments of higher education. At least six of the states researched detailed 
agreements on vocational-technical course and credit transfer. 

When investigating secondary/postsecondary programs with occupationally related curricula, 
Long et al. (1986) reported that articulation planning efforts were currently underway in 29 states. 
These articulation efforts included vocational-technical schools, high schools, community colleges, 4- 
year universities, and businesses as joint participants. Articulation efforts varied greatly from 
institution to institution, based upon numerous variables such as institutional size, resources, program 
goals, and philosophies. 

Long et al. also identified two primary models of secondary/postsecondary articulation efforts: 
1) time-shortened models and 2) advanced skills models. They reported that the purpose of the time- 
shortened model is to eliminate unnecessary educational redundancy by granting advanced placement 
to secondary students entering a 2-year college program. The National Council for Occupational 
Education (1989) reports that the advanced placement models are the most common and least 
complicated form of articulation efforts since fewer curriculum changes are required. The majority of 
the time-shortened programs are course-based, through coordination of syllabuses or task lists (Long 
et al., 1986). 

In contrast, advanced skill models focus on delivering a more advanced and concentrated 
content. Generally, advanced skills models are categorized as core curriculum, which is widely 
recognized as tech-prep or vocational 2 + 2 programs (Long et al., 1986). Tech-prep programs 
provide advanced technical skill preparation in the applied sciences, mathematics, and technical 
writing. The 2 + 2 program is a 4-year articulation plan which includes 2 years of secondary 
training and 2 years of training at postsecondary institutions (McKinney et al., 1988). 

Articulation has been found to be a viable strategy in addressing both student and interagency 
concerns. The benefits of articulation identified for students include allowing students to gain 
advanced standing or college credit, reducing instructional repetition, and providing more course 
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options. Articulation further motivates students to continue their education through recognizing 
advanced credit and serving as a career exploration strategy (Fadale and Winter, 1987). In addition, 
McKinney et al. (1988) suggested that articulation encourages student development by improving 
programing and facilitating student transition from one educational setting to another. 

From the educational institution perspective, articulation provides strategies to address 
community service obligations and the opportunity to avoid costly duplication of facilities and 
resources. By exemplifying cooperation among institutions, articulation subsequently serves as a 
public relations tool and demonstrates efficient use of taxpayer money (Fadale and Winter, 1987). A 
frequently mentioned benefit of articulation for secondary institutions is image enhancement of 
vocational education as a result of the visible link with higher education institutions. This link was 
further found to enhance recruitment of high-quality students into vocational programs. Increased 
enrollment, higher retention, curriculum revision, and program improvement have also been listed as 
institutional benefits (McKinney et al., 1988). 

As noted by McKinney et al. (1988), despite the number of articulation programs in 
existence, there is relatively little evaluation information regarding articulated program effectiveness 
or the appropriate processes for implementation. They investigated the degree of institutional success 
in achieving articulation program goals. In that study, it was found that the most commonly accepted 
goals of vocational-technical education articulation programs were increasing service to students, 
program improvement, increasing student retention, and increasing service to employers. Results 
indicated that institutions had the most success meeting the program improvement and increased 
service to student goals. 

Several studies have sought to isolate the characteristics of successful articulation programs. 
Administrative commitment from the top, leadership, faculty involvement, communication, and 
common goal focus have been identified as essential elements (McKinney et al., 1988; Long et al., 
1986; and Farland and Anderson, 1988). Leadership and commitment from the top are essential to 
articulation programs because the leaders serve as guiding, motivating, and driving forces, and lend 
political, emotional, and financial support to faculty. Without common goal focus, communication, 
and faculty interaction, it would be impossible for articulation efforts to develop. 

ISSUES TO INVESTIGATE 

For those conducting the National Assessment, the principal issues warranting investigation 
may well be more than can be reasonably addressed, given the parameters set forth in the federal 
legislation. However, certain of these issues are resolute and are based on the assumption that 
various aspects of the federal legislation that can lead to secondary/postsecondary articulation will, in 
fact, do so. 

These issues will include 1) a description of the articulated programs being assessed, 
including the degree to which the federal requirements result in articulated programs and student 
participation in such programs, 2) the outcomes and effectiveness of articulation between secondary 
and postsecondary program, and 3) the factors which underlie exemplary articulation programs. 
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Articulated Program Descriptions 

As was briefly discussed in the background literature review, articulation initiatives can be 
implemented in various ways. Regardless of whether or not the National Assessment researchers opt 
to operationally limit the definition of articulation (further discussed under the section "Problems to 
Anticipate") it would be useful to become more familiar with the articulation models that are the 
intent of the investigation. 

Characteristics and issues related to the articulated programs that could be described include 
the following: 

1. Type of articulation model (e.g., tech-prep, time-shortened). 

2. Goals of the program (e.g., student retention, program improvement/educational 
excellence, and increasing the number and quality of graduates available for business 
and industry). 

3. Program context variables (e.g., student characteristics, including ability levels; 
school characteristics; and community and economic characteristics). 

4. Governance of the program. 

5. Staffing and staff development. 

6. Planning and evaluation. 

7. Curriculum development. 

8. Program support services. 

9. Individuals' (including teachers, administrators, counselors, parents, students, and 
employers) understanding of, support for, and attitude toward the program. 

10. Marketing of the program. 

11. Human and financial resources. 

12. Related operational issues (e.g., the articulation agreement, facility needs, and 
accreditation). 

13. Costs and cost-effectiveness of the program. 

In regard to operational issues, considerable attention should be given to describing how 
"common ground" is decided upon in the articulation agreement. At the secondary level, program 
outcomes are often associated with competency development, whereas course or degree completion is 
most often the targeted outcome at the postsecondary level. 
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In the Perkins Act, articulation agreements are defined as M a commitment to a program 
designed to provide students with a nonduplicative sequence of progressive achievement leading to 
competencies in a tech-prep education program." The way program outcomes at the secondary and 
postsecondary levels are meshed will be a major challenge in developing a successful articulation 
program and are of considerable importance to vocational education practitioners. 

Although informative, a determination of the number and type (i.e., occupational area) of 
articulation programs may be difficult to achieve. It is possible that not all articulation efforts at the 
local level would be reported to the sole state vocational education agency. 

In regard to types of articulated programs, it may be that health occupations programs are 
more likely to be articulated than are trade and industrial programs. Here again, identifying 
articulation programs by type may be difficult. However, the query should be made, on a state-by- 
state basis. The same rationale holds true for the number of students entering and completing an 
articulated program; that is, such data are an indicator of success or failure but are difficult to obtain 
on a comprehensive basis. 

Outcomes and Effectiveness of Articulated Programs 

The outcomes and effectiveness of vocational education programs are historically debated 
topics. No doubt there will also be debate on the outcomes and effectiveness of articulated vocational 
education programs. Numerous issues can and should be raised in relation to the outcomes and 
effectiveness of articulated efforts between secondary and postsecondary vocational education 
programs. However, the time frame of the study, discussed later in the paper, may be extremely 
limiting when investigating this issue. These questions can be asked in the contexts of the students 
and the institutions involved in articulation programs. 

From the student perspective, the outcomes which might be assessed are, to a large extent, 
traditional and might be categorized as labor market, learning, and ancillary outcomes. They include 

1. Employment (including job-related employment) and unemployment; 

2. Course utilization rates (CURs); 

3. Wage rates and earnings; 

4. Employee satisfaction; 

5. Employer satisfaction; 

6. Occupational skill attainment; 

7. Academic skill attainment; 

8. Retention and attrition; 
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9. 



Continuing education (including bachelor of science degree programs); 



10. Aspirations; and 

1 1 . Attitudes and values . 

In assessing the above outcomes for students enrolled in articulation programs, it would also 
be interesting to compare these outcomes with proups of students who do not participate in an 
articulated program. Are either groups of students at a labor market or educational disadvantage? 

In regard to program and institutional outcomes, the following questions can be asked: 

1. Do student enrollments increase at the secondary and postsecondary institutions? 

2. Are better quality students attracted to articulated programs? 

3. Have program content and standards improved? 

4. Are facilities and equipment being better utilized? 

5. Is there actually less duplication of effort between the secondary and postsecondary 
institutions? 

6. Do articulated programs lessen the attrition problem in the postsecondary institution? 

The last question is particulary important in view of the fact that students who take only a few 
courses are no more likely to be employed or earn higher wages than those who take no courses at all 
(Goodwin, 1989). Obviously, the extent to which each question can be answered is contingent on the 
availability of baseline information. 

Exemplary Articulation Programs 

As in the previous discussion on the description of articulated programs, this portion of the 
assessment would require an extensive analysis of inherent contextual and process variables. These 
variables would include those listed in the previous section, but they would be investigated in a 
manner that would answer "how" and "why" questions. In essence, the assessment of exemplary 
articulation programs involves finding the outcome and determining "why" versus targeting the 
outcome and determining "if." 

Obviously, identification of successful articulated programs would require an investigation of 
the factors which underlie such programs. The programs studied should be identified so as to 
accommodate, at a minimum, diversity in regard to population, economy, geographic region, 
vocational education governance structure, and federal funding level. 

In addition to investigating those factors inherent in the articulated program, the National 
Assessment researchers should take into account related circumstances and forces that might have 
influenced development and implementation of the exemplary articulated programs. For example, 
what influence has the state accreditation agency exercised, and what role have enrollment trends at 
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the secondary or postsecondary institutions had on development and implementation of articulation 
agreements? Also, who, if anyone, has wielded the most influence in the articulation initiative (i.e., 
the secondary or postsecondary educational entity), and to what extent do changes take place in each 
institution? 

PRINCIPAL DATA SOURCES AND METHODS OF INQUIRY 

Multiple data sources and methods of inquiry are required to investigate the issues outlined in 
the previous section. For example, case studies and surveys would serve as complementary research 
methods in investigating local sites that have initiated articulation linkages between secondary and 
postsecondary vocational education programs. Recommended data sources and methods of inquiry 
are presented in the following sections. 

Previous/Ongoing Research and Related Literature 

Previous research, ongoing research, and related literature are important data sources in the 
preliminary design of this study. As has been shown in the abbreviated literature review presented in 
this paper, articulation between secondary and postsecondary vocational education programs can by 
no means be considered a new direction for vocational education as a result of the Perkins Act. 

An extensive review of the research and related literature on articulation is recommended to 
augment those areas addressed in this paper. When conducting the review of research and related 
literature, the National Assessment design team is advised that they are likely to find that a 
preponderence of the literature focuses on the operation of articulation programs and that the literature 
base on articulation outcomes is limited. Nonetheless, a more complete understanding of the issue 
will undoubtedly result from these efforts. 

The following are recommended as possible sources of relevant research and related literature 
on articulation between secondary and postsecondary vocational education programs: Resources in 
Vocational Education (RIVE), Resources in Education (RIE) t Current Index to Journals in Education 
(CUE), and Dissertation Abstracts. 

In regard to ongoing research, the National Center for Research in Vocational Education 
(NCRVE) is currently funding two projects related to articulation and, more specifically, to tech-prep. 
These are "Factors Influencing Tech-Prep Planning and Implementation" and "An Information 
Exchange on Tech-Prep Programs." Both projects will undoubtedly be of value to the National 
Assessment. 

Surveys 

In many large-scale assessments, surveys are typically a default method of inquiry. It is 
recommended that the articulation aspect of the National Assessment also be partially conducted 
through survey methods. Such methods will result in a "sketch" of the articulation undertaking drawn 
from a larger population. However, the outcomes obtained probably will not penetrate very deeply 
below the surface of the issue and measurement error may be a problem. There also may be a protest 
of "response burden" at the state and local levels. 
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There is uncertainty at this time as to whether or not the 50-state survey conducted by the 
NCRVE included an articulation component. (Public domain products are scheduled for release 
sometime this year). Regardless of the inclusion of an articulation component in the survey, both the 
NCRVE and the previous NAVE researchers possess valuable experience and insight in regard to this 
particular methodology. 

It would seem reasonable to assess effects of the Perkins Act on articulation across all states. 
Assessing effects at the local level is more difficult, due to extreme variability in articulation program 
characteristics and issues. At the local level, investigating secondary and postsecondary articulation 
practices is feasible only by using a sample of eligible recipients. 

Case Studies 

In an effort to answer "how" and "why" questions and allow those who are being studied to 
speak for themselves, focus on contemporary events in articulated programs is highly recommended. 
As in previous national assessments of vocational education, case study methods should again be 
relied upon as a primary method of inquiry. 

Ideally, the case study sites should be selected using predetermined criteria. They should 
reflect diversity in regard to geographic region, economy, vocational education governance structure, 
federal funding level, and population diversity. To develop a more in-depth understanding of a 
limited number of vocational education delivery systems, it is also recommended that a number of 
"core states" be identified (i.e., states which are involved in as many components of the National 
Assessment as possible). 

Documentation 

Documentation, or documentary analysis, is often viewed as an integral aspect of case study 
methodology. For the purposes of this study, documentation is viewed as a source of information in 
two contexts, the first of which is the case study of the exemplary sites. 

When visiting the sites, the study team will want to review carefully a variety of documents 
related to the articulation initiative, including, but not limited to, letters, memoranda, and other 
communiques; agendas, announcements, and minutes of meetings; previous studies or evaluations of 
the site; mass media materials; and articulation agreements. 

The states themselves are the second context in which documentation is a viable data source. 
The particular documents that should provide considerable insight into the articulation issue are tech- 
prep education RFP's, tech-prep proposals, and local applications or plans. The latter would 
document the extent to which eligible recipients are using Title II funds for tech-prep education. The 
former should clearly illuminate the manner in which Title HI, Part E funds are being expended. 

Two other documents which are likely to provide an indication of the effects of federal 
requirements on intrastate articulation efforts are state plans and federal performance reports. 



52 

55 



Transcript Analyses 



Transcript studies have been shown to be very useful for analyzing secondary and 
postsecondary participation patterns in vocational education and for estimating course utilization rates 
(Campbell, Orth, and Seitz, 1981; Hoachlander and Choy, 1986; and Tuma, Gifford, and 
Hoachlander, 1989.) 

In regard to participation patterns, the National Assessment researchers could use the 
taxonomy developed for the previous national assessment. To compute CURs, all courses listed on 
the transcripts of the articulated program completers could be fed through the National Occupational 
Information Coordinating Committee matching matrix and mapped to the occupations they hold. 

Wage Record and Other Existing Data Bases 

Vocational educators are demonstrating a renewed interest in using the unemployment 
insurance wage-record system as an evaluation methodology. Use of wage records may be an 
alternative means for assessing measures for the labor market outcomes of articulated program 
completers. 

When articulated program completer records are merged with unemployment insurance wage 
records, post-program earnings histories can be examined by participant characteristics, vocational 
program, geography, Standard Industrial Classification status, or any combination of these variables. 
A comparison group could also be included as part of the evaluation. Other advantages of the 
methodology: 1) The data are more accurate than self-reported data; 2) Data are obtained for a larger 
percentage of students than can be collected through followup mail surveys; and 3) Longitudinal 
followups can be implemented at reduced costs and in less time. 

In spite of its advantages, this methodology is problematic. First, if the issue of articulated 
program completer earnings is being investigated, it may be difficult to differentiate part-time 
employment or employment initiated after the reporting period began, since the rate of pay is not 
always included as part of the employer report. Second, the Standard Industrial Classification reflects 
the nature of the industry versus the specific occupation in which the articulated program completer 
would be employed. This would be of concern in determining the "relatedness" of the articulated 
program to job placement. Third, the methodology is contingent upon a fairly sophisticated, 
automated student record-keeping system. (For a thorough review of the uses and limits of 
unemployment insurance wage records, see Stevens (1989), Using State Unemployment Insurance 
Wige-Records to Evaluate the Subsequent Labor Market Experiences of Vocational Education 
Program Leavers.) 

Based on the principle of using a common element to match records from two data bases, 
higher education data bases might also be assessed to determine the extent to which articulated 
program completers continue on in postsecondary education and complete the requirements for a 
bachelor of science degree. 

PROBLEMS TO ANTICIPATE 

As in any assessment in an educational setting, there are numerous methodological concerns 
as well as problems specific to the National Assessments study of this topic. Although none appears 
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to be insurmountable, all warrant attention during the conceptualization and pre-implementation stages 
of this assessment. 

Defining Articulation 

In assessing the effect of federal requirements regarding articulation between secondary and 
postsecondary vocational education programs, the National Assessment researchers are faced with the 
difficult task of defining or setting an operational limit on the concept of articulation. This task is 
difficult in that articulated programs between secondary and postsecondary vocational education 
programs vary widely in operational details and primary goals. For example, the three basic types of 
articulation programs are time-shortened, tech-prep, and 2+2 (McKinney et. al, 1988). These types 
of articulation programs can be conceptualized as falling on a continuum, with time-shortened 
programs being the least complex and 2+2 programs the most complex. Numerous other variations 
of articulation programs fall along the continuum and undoubtedly outside the types of articulation 
programs referenced (e.g., 2+2+2). 

Consequently, a critical question that must be resolved when designing this study is: What 
type of articulation program is being investigated? (Or will it be the the purpose of the assessment to 
investigate articulation programs along the continuum?) In that tech-prep education is an integral 
aspect of the Perkins Act in both Title III, Part E and an allowable expenditure at the local level, the 
study design may need to focus on that type of articulation program. 

Time Frame of the Assessment 

The Perkins Act requires that a final report, summarizing all studies and analyses completed 
after the assessment, shall be submitted to the Congress on or before July 1, 1994 [Sec. 403(c)(2)(B)]. 
As a result, 1992-1993 is probably the last school year for which data related to articulation initiatives 
could be collected. In regard to only one phase of an articulation program (i.e., preparation for 
program implementation), Belcher (1991) stated that "the first decision by administrators who propose 
to implement tech-prep associate degree programs should be to allow at least 12 months to conduct 
research, identify the issues, and address as many as possible prior to implementation." Obviously, 
the time frame mandated in the legislation will be problematic, particularly in regard to assessing 
articulation outcomes. 

Undoubtedly, countless articulation efforts will be undertaken as a direct result of the tech- 
prep language in the legislation. The question is: To what extent can true outcomes be assessed? 
(The assumption is that the amount of time an articulation program has been in operation will be a 
factor in these outcomes.) It seems reasonable to believe that relatively few of the articulated 
programs will have been in operation long enough to resolve program difficulties and produce 
optimum results. 

1990 Perkins Act Articulation Initiatives 

As can be seen in the literature, articulation between secondary and postsecondary vocational 
education has been implemented in various forms prior to the passage of the 1990 Perkins Act. For 
the most part, articulation initiatives undertaken before the 1990 Act are not a result of federal 
legislation. 
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The obvious problems are 1) separating articulation initiatives that were implemented as a 
result of the federal legislation from those which already existed and 2) accounting for the effects of 
the prelegislation articulation programs on 1990 Perkins Act articulation programs. Here again, it is 
reasonable to assume that the time period over which the articulation program has been operable and 
articulation outcomes are correlated, as are previous experience with articulated programs antf 
articulation outcomes. 

If the purpose of the study is to assess only the effect of the federal requirements in the 1990 
Perkins Act on articulation between secondary and postsecondary vocational education programs, it 
will be necessary to incorporate time-oriented items on the data collection instruments to establish the 
start-up date of the program. At the same time, it would be important to gain a historical perspective 
of the local site or the state regarding articulation. This also could be ascertained through appropriate 
items on the data collection instruments. 

State Resources and Data Bases 

Massive state educational budget deficits and reduced discretionary funding authority under 
the 1990 Perkins Act are forcing many states to reduce vocational education staffing. If state 
vocational education personnel are perceived as a resource for collecting or coordinating data 
collection, a potential problem exists. At a minimum, collecting and reporting data on a statewide 
basis can be expected to be a tedious undertaking. 

Since articulation initiatives have historically been associated with low enrollments and scarce 
financial resources, it may be that articulation arrangements are catalyzed by those factors rather than 
by federal requirements. Differentiating the effect and determining the interaction of federal 
requirements and limited funding may be problematic. 

If any portion of the assessment is dependent on widespread use of vocational education data 
bases at the state level, problems are likely to be encountered due to incomplete and, in some cases, 
nonexistent data bases. Goodwin (1989) reported that the principal problem with measuring labor- 
market outcomes on a state or programmatic basis is the inadequacy of data. Student followups are 
frequently characterized by unacceptably high levels of nonresponse or "status unknown" reports. 

(Further information and a historical review of efforts to collect national information are 
reported by Hoachlander (1989) in an insightful document, National Data Needs for Vocational 
Education.) 

Methodological Concerns and Technical Considerations 

Many methodological concerns or technical considerations are associated with each method of 
inquiry. Although these concerns and considerations do not constitute irresoluble problems, they do 
warrant attention and require time to resolve. For example, if a mail survey is used to gather data at 
either the state or local level, consideration must be given to the type of sample, question wording, 
followup procedures, reliability of the instrument, amelioration of measurement error, threats to 
external validity, pretesting, and nonresponse, as well as to the design, layout, and format of the 
questionnaire. 
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Each of these methodological concerns or technical considerations must be thought through in 
terms of the underlying issues. For example, in sampling, considerations should include whether or 
not to use a probability sample, who makes up the sample frame, and the size of the sample. 

The same rationale holds true regardless of the method of inquiry; that is, methodological 
concerns and technical considerations must be adequately addressed to ensure valid, reliable, and 
practical outcomes. The purpose here is not to present a comprehensive review of such concerns and 
considerations, but rather to remind the National Assessment researchers of the many decisions that 
underlie an assessment of the effects of federal requirements on articulation and to caution them that, 
in many cases, it often takes longer than anticipated to resolve these issues. 

SUMMARY 

Based on data collected from key state officials responsible for secondary and postsecondary 
occupational education programs and data provided by state directors of vocational education to 
identify articulation efforts, Bender (1973) concluded: 

Society, its legislative representatives, and professional educators have come to realize that 
each component unit of the educational system must be in its place, interconnected, and joined 
to form a true continuum. This does not mean the loss of special purpose, mission, or 
identity of each unit. It does mean, however, that the educational delivery system cannot 
have individual components going in opposite or independent directions (p. 37). 

Confronted with the challenge of developing a highly educated, world-class work force, the 
above conclusion takes on even greater significance nearly 20 years later. It is absolutely imperative 
that secondary and postsecondary vocational education institutions work together to develop and 
sustain quality programs to serve the majority of Americans who are already working or who can 
potentially enter the work force. 

Obviously, assessing the effects of federal requirements on articulation between secondary and 
postsecondary vocational education programs is an extremely complex undertaking. Nonetheless, 
such an assessment is needed not only to report to the Congress but also to provide vocational 
practitioners with an indication of how well they are doing in preparing the work force of the future. 
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Views on Assessment of the 1990 Perkins Act 



Joseph W. Grimsley* 



This paper discusses three major areas of change in the 1990 Perkins Act: targeting 
disadvantaged populations, performance measures, and tech-prep programs. 

TARGETING DISADVANTAGED POPULATIONS 

With the new targeting criteria in the 1990 Perkins Act, the postsecondary system will face a 
difficult adjustment that will delay the effectiveness of the Act. Heretofore, most of the Perkins funds 
were generally blended with state and local vocational education funds without individual knowledge 
of recipients of services. The 1990 Perkins Act requires that future grants to local schools and 
colleges come in narrowly focused grants, with the individual recipients identified by auditors and 
assessors. 

The issue of less effectiveness involves increased recordkeeping and more categorical program 
management. It is highly likely that many operators will narrowly interpret the use of vocational 
education funds and focus on vocational education rather than on a broad-based educational program 
with math, science, English, and social studies in the forefront. It will take a year or two to adjust to 
the new criteria and to determine what is and is not permitted. The more specifically targeted 
disadvantaged population will seem more isolated from the mainstream than in the past as they are 
singled out once again as different. (Note the stigma attached to free lunches, food stamps, etc.) 
From the author's experience, specific dollar targeting lessens involvement of the target populations in 
the overall school efforts in that members of those populations are counted off for separate attention 
(for recordkeeping purposes). 

Targeting disadvantaged students is not, in itself, a bad idea. However, it will focus the 
dollars available from the Perkins Act on the most difficult to educate and train. It probably will cost 
more per student than before, cost the program overall public support, and produce less positive 
results per dollar invested. 

Therefore, concerns the National Assessment should address include: 

1 . What is the degree of shift from previous recipients to new recipients? 

2. What are the per student costs compared to prior years? 

3. What level of disadvantaged students will now be served? 

4. Does the new program target most severely disadvantaged students or others? 



♦Joseph W. Grimsley is President of Richmond Community College, Hamlet, North Carolina. 
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5. What is the impact on overall vocational education when disadvantaged targeting 
requirements shift resources away from general support? 

6. Will there be a smaller overall vocational education program because of the new 
Perkins Act's shift away from general support? 

7. Will targeted disadvantaged programs offer a good tech-prep approach with 
requirements for integrating academics and vocational education? 

Data Sources 

Most schools and colleges have formal data bases required by the state education or 
community college agency. These data bases are usually called uniform accounting and student PTE 
accounting systems. Most colleges also have record systems which list the students by course. The 
students are often recorded by their disadvantage, at least in terms of eligibility for financial aid, 
specific handicap, and participation in other programs which are open to disadvantaged students. 
Most of the local college record systems are fed by magnetic tape to the state agency, which can 
generate state-level summaries of most required data. 

Modern student record systems are very sophisticated. They include many personal 
characteristics, placement test scores, the handicapping condition, transcripts to date, the educational 
plan, special needs, and an increasing amount of academic tracking data regarding classroom 
performance during a quarter or semester. Such systems are coming into use in both public schools 
and community colleges. North Carolina is already using these record systems. They facilitate the 
ability to report on any type of student in response to new data requirements. It would be shameful 
to ask colleges and school systems to buy large new computer systems and write expensive software 
programs to replicate, in large part, data that are already being collected. Lack of federal vocational 
education administrative funds at the local college and school level leads to negative feelings if large 
new reporting systems are required. Administrators' loss of morale may mean that students may not 
receive the level of attention they deserve. 

PERFORMANCE MEASURES 

The overall issue of performance measures raises concern about increased data collection 
requirements, rather than fear of accountability. In the North Carolina Community College system, 
there is a set of accountability capabilities that should be acceptable for National Assessment 
purposes. For example, North Carolina has established " critical success" factors which measure 
many overall aspects of program effectiveness. All southern states belong to the Southern Association 
of Colleges and Schools (SACS), which accredits colleges and establishes extensive criteria for 
accreditation. SACS also requires campus-level "institutional effectiveness plans" which are annual 
self-studies that encourage ongoing institutional effectiveness throughout the normal 10-year 
accreditation cycles. 

North Carolina may be more advanced than some states in that it has a strong state-level 
oversight operation. However, North Carolina, South Carolina, Kentucky, Florida, Virginia, and 
some other states have sophisticated reporting systems involving many measurable factors. These 
measurable factors far exceed those included in the Perkins Act. 
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For the National Assessment, there is an issue related to management efficiency and including 
the concerns of states and locales in the assessment. Heretofore, state and local concerns seemed to 
mean very little to Congress. Now, states and local governments are heavily involved in educational 
reform and the Congress ought to understand that the Perkins Act is a small part of the overall 
movement. What is happening overall in vocational education? Perhaps a great deal is being done 
for the target population which is not readied by the Perkins Act, since states know that Perkins 
money cannot reach all die target populations. The concern is that the National Assessment will 
overlook both die vocational education offered to disadvantaged populations with funds other than 
Perkins and the performance assessment systems developed with other funds. (For example, North 
Carolina Community Colleges receive $450 million from the state legislature.) An issue for 
assessment is: Will standards and performance measures help improve vocational education and force 
programs into the tech-prep format? 

ASSESSMENT OF TECH-PREP/ ASSOCIATE DEGREE PROGRAM 

Dr. Dale Pamell, president of the American Association of Community and Junior Colleges 
(AACJC), developed the thesis for tech-prep in his pioneering book, The Neglected Majority, 
published by AACJC in 1985. The premise was that only a minority of students pursue the perceived 
high school track of excellence, college-prep. However, the nation needs up to 75 percent of all high 
school students to be prepared for a postsecondary education to create a world-class work force in 
America. Parnell recommended focusing the neglected middle, average high school students toward a 
technological education, with the goal of achieving the associate degree from a community or junior 
college, as a minimum. 

Parnell's premise further established that most American dropouts come from this neglected 
middle majority, not from the learning disadvantaged or college-prep students at either end of the 
spectrum. Research on dropouts has indicated that lack of purpose clouds the middle students' view 
of educational purpose and leads them to give up on purposeless education. Tech-prep is designed to 
focus these middle students early in high school toward a worthwhile, financially rewarding, and open 
job market for community college-trained technicians (i.e., nurses, computer operators, bookkeepers, 
policemen, EMH technicians, mechanics and machinists, etc.). 

The tech-prep/associate degree program is one of the three most important changes in the 
1990 Perkins Act. Since tech-prep is the first truly new educational content title in vocational 
education in recent history, it is very important that the U.S. Department of Education take great care 
to focus more deeply on program content than on who receives the services. 

It is the author's premise that tech-prep can be the most significant development in high 
school education in the past 30 years (since creation of the college-prep track). It will meet this high 
goal only if the funds allocated in each state and received by its local schools and colleges truly 
support integration of academic and vocational education. 

Integration of math, science, English, and social sciences into a program with emphasis on 
vocational education is a vital component of the tech-prep/associate degree program. Under this 
program, students would be expected to take grade-level academics (the same as required for college- 
prep students in most states) throughout the high school career, and these grade-level academics 
would be taught with the goal of enrollment at a community college. North Carolina requires 20 
courses for graduation, including 14 academic courses: three units of math (in tech-prep these are 
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algebra I and II and geometry); 4 years of English; three sciences; three social sciences; and one 
physical education/health course. In the vocational education program, if these are given equal 
weight with the other six required courses for graduation, the vocational education program will be 
effective even though it involves no more than the six vocational courses. The most important part of 
the integrated education program is the 14 basic required courses. 

The tech-prep/associate degree program also attempts to increase the math, science, and 
communications instruction in vocational education courses, trying at all times to teach the academics 
appropriate to the grade level so that upon graduation the student will be qualified to enter and 
perform effectively in a collegiate associate degree program. A premise of the tech-prep/associate 
degree program is that the middle majority of high schoolers can be focused toward postsecondary 
education, will work hard to achieve at grade level, and will get the formal education required for a 
carefully articulated high school/community college degree program. It means that the high school 
student will have vocational education courses which build academic abilities rather than job-entry 
skills; the community college will provide higher level job-entry skills as the student pursues the 
associate degree. 

Another premise is that algebra and geometry courses should not be altered for vocational 
education students. In upper grades, applied math and other applied courses can be added as part of 
the vocational education experience, but never at the expense of the formal math and science courses 
also required for college-prep students. The major concern about the Perkins tech-prep effort is that 
it will be captured by high school vocational education directors and teachers who will continue to 
isolate vocational education from the academic mainstream. Disadvantaged students will be the big 
losers in this scenario. The high schools must convince themselves and these students that the 
students can learn upper-level math, science, and communications, and that these academic skills are 
the basis of good vocational education and the future associate degree. 

Finally, the legal requirement for a collaborative effort between public schools and 
community colleges to articulate the requirements of the associate degree with high school vocational 
education should be the means by which the appropriate programing takes place. If this is not done, 
the program is likely to fail. This should be an item in the National Assessment. 

Many aspects of this critical new innovation in American education, tech-prep, should be 
assessed, including: 

1. The continuity of learning from the high school level to the community college level 
to entry and achievement in the job market. 

2. The integration of academics in tech-prep programs (e.g., how many vocational 
students take higher level math such as algebra, geometry, and calculus). 

3. Context-based teaching and learning. (Are applied academics and basic skills 
integrated into vocational education courses; are grade-level communication (English) 
skills and math skills required in papers and projects involving different vocational 
sequences?) 



62 

65 



4. Competency-based teaching techniques, to determine if vocational education students 
are really learning the full set of competencies in a field rather than only the 
introductory and hands-on skills. (Are higher-order thinking skills utilized?) 

5. The depth of the relationship between public schools and community colleges in 
planning, articulating, improving, and operating a true tech-prep partnership. Items 
should include: 

a. Is there a written agreement between the two boards? 

b. Are chief executive officers involved and committed? Do they meet? 

c. Are joint steering committees and joint articulation committees active? 

d. Are both staffs adequately oriented toward the purposes of tech-prep? 

e. Are math, science, English, and social science courses articulated with 
appropriate vocational courses? Are teachers and department chairs from both 
systems involved? 

f . Are students actively recruited to be part of the tech-prep/associate degree 
program? 

g. Are tech-prep student records monitored carefully for course of study? 

h. Are counselors trained and committed to tech-prep students and programs? 
Do counselors have associate degrees? 

i. Are annual reviews and revisions made, results reported publicly, and 
programs changed to respond to shortcomings? 

j. Are vocational courses upgraded regularly by articulation committees for 
content and equipment? 

k. How do tech-prep students do on community college placement tests? 

1. How do tech-prep students do in community college courses? 

m. What are tech-prep students* graduation rates? 

n. How do tech-prep students do in placements in job fields after receiving the 
associate degree? 

6. Whether or not the tech-prep/associate degree program develops into a respected 
program of educational excellence on par with the college prep program. (Paraell has 
repeatedly stated that vocational education has one major defect: prestige deficiency. 
That lack of prestige has robbed vocational education of its credibility with business 
and industry, postsecondary institutions, and the general public. The National 
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Assessment should try to determine whether the new Perkins Act, with emphasis on 
integration of academics and the tech-prep/associate degree program, will really bring 
about a vocational education revival and increased prestige. This could be done 
through national polls. The prime target should be employers, community colleges, 
teachers, administrators, and the students themselves.) 

The Tech-Prep Target Population 

Tech-prep is designed to reach the "Bart Simpson" academic underachiever. It is important 
for the National Assessment to evaluate whether tech-prep reaches this target population. The 
students usually take general education and are characterized as 

1. underachieving in school (based on known abilities); 

2. uninterested in schoolwork (take easy courses); 

3. high potential as dropouts; and 

4. underprepared as future community college students. 

These general education students represent a majority of high school students and will make 
or break America in the next century. The Congress has recognized the importance of raising the 
expectations of this majority and has focused the tech-prep title in this new direction. We need to 
know 

1. The effect of tech-prep on dropout rates among target students; 

2. The trend in taking higher math and English and technical courses among the target 
students; 

3. The target students' retention rate after enrollment in community colleges; and 

4. The status of those middle students not taking tech-prep. 

Community College Articulation Role 

Tech-prep calls for close collaboration between community colleges and public schools in 
order to articulate the curriculum from the high school level to the 2 years required for an associate 
degree (in applied science). The tech-prep title in the new Perkins Act calls for formal agreements 
between the colleges and schools. The congressional intent was to ensure that a large majority of 
American students meets the educational requirements of 2- or 4-year colleges, and that 
undermottvated and underachieving students are prepared to achieve education at least to the 2-year 
college level. Implications for the National Assessment include: 

1. Does the tech-prep program run parallel to college-prep but not replace it? 

2. Does it build high school academic-core education for entering 2-year colleges? 
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3. 



Does it lead to an associate degree program? 



4. Does it include strong technical courses, including math and science concepts built 
around clusters of occupations at the community college level? 

5. What is the level of community college participation in planning, designing, and 
monitoring the local tech-prep programs? 

CONCLUSION: INTEGRATING ACADEMIC AND VOCATIONAL EDUCATION 

It appears obvious that Congress intended that some form of standards arise from this 
legislation that would help move toward a national consensus on the meaning of integrating academics 
and vocational education. Two types of approaches are likely to develop. Which approach dominates 
in the next 4 years is a subject for study. The "separate but equal" approach involves offering the 
academic core curriculum "straight up": English, algebra, science, and social studies, with no 
compromise or direct technological applications. The "applied" approach attempts to build high-level 
academic competencies into high-level technology courses with the goal of teaching academics and 
technology in the same courses. 

In the National Assessment, an effort should be made to determine which of these approaches 
was used to best advantage in reaching the underachieving "middle" students. Both approaches have 
a great deal of potential, but community colleges will probably prefer the academic core curriculum 
over applied academics, while high school vocational educators will prefer the applied academics. 
Community colleges will want strong stand-alone math and science skills as the base for higher 
education courses. High school vocational education leaders will probably want a more limited view 
of academics and more interest in vocational competency and completion of the vocational course at 
the end of high school. Questions for the National Assessment include: 

1. What is the dominant approach to integrating academics and vocational education in 
tech-prep programs? 

2. Does one approach prove more motivating and academically effective than the other? 

3. Does one approach make a better community college completer than the other? 

[Note: A recent book which captures the national tech-prep experience to early 1991 is Tech 
Prep Associate Degree: A Win-Win Experience. It was written and compiled by Dan Hall and Dale 
Parnell and printed in Dallas, Texas, by the Center for Occupational Research and Development.] 
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Analyzing Federal Expenditures for Vocational 
Education Program Improvement 



Gary Hoachlander* 



INTRODUCTION 

Congress has traditionally asked three questions about federal spending for vocational 
education: 

• What does vocational education cost? 

• What do federal dollars buy? 

• What impact does federal spending have on vocational education programs? 

These seemingly simple but important questions have proven exceedingly difficult to answer. 
This paper explains why and suggests some possible approaches the National Assessment of 
Vocational Education might take to inquire about federal spending for vocational education. 

WHAT DOES VOCATIONAL EDUCATION COST? 

Federal Spending as a Percentage of Total Spending for Vocational Education 

Federal spending represents a rather small fraction of total spending for secondary and 
postsecondary vocational education in the United States. The figure cannot be determined precisely 
but probably lies in the range of 5 to 10 percent of total expenditures. This estimate is based on 
figures supplied by the states to the federal government during the 1970s and early 1980s, which 
purported to show that federal spending amounted to about 10 percent of total spending for vocational 
education. 1 As federal spending for vocational education has remained relatively constant at about $1 
billion in recent years, the effects of inflation have eroded the relative share of the federal dollars. If 
the earlier estimates of state and local spending for vocational education were at all accurate, today 
the federal share lies closer to 5 percent than to 10 percent. 



*Gary Hoachlander, Ph.L., is president of MPR Associates, Inc., a consulting firm specializing in 
public sector management, planning, and research. He is also director for National Planning and 
Evaluation at the National Center for Research in Vocational Education at the University of California, 
Berkeley. The views expressed in this presentation are his own and not necessarily those of the National 
Center for Research in Vocational Education or the U.S. Department of Education. 

l See National Center for Education Statistics, The Condition of Vocational Education, Washington, 
D.C.: U.S. Government Printing Office, July 1981; Project Baseline, Learning a Living Across the 
Nation, Flagstaff, Arizona: Northern Arizona University, Vol. 1-4, 1971-1975. 
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But how accurate were those estimates? No one knows for certain, and the federal 
government ceased collecting data on state and local spending for vocational education when the 
Vocational Education Data System (VEDS) was suspended in December 1983. 2 The reason the 
estimates are suspect is twofold. First, most local school districts and postsecondary institutions do 
not maintain financial accounts by program. Secondary and postsecondary education agencies in this 
country can no more tell what is spent on vocational education than on math, science, or English, 
Second, assuming that accounts could be maintained programmatically, determining what should be 
counted would remain a problem. The programmatic boundaries of vocational education are 
somewhat fuzzy, making precise estimates of program spending impossible. Both of these issues are 
worth examining in more detail. 

Local Budgeting and Accounting Practices 

Budgeting and accounting procedures in secondary and postsecondary education in the United 
States typically classify spending by -object" of expenditure. 3 Objects of expenditure include salaries, 
benefits, supplies, equipment, travel, outside services, and so forth. Additionally, these general 
objects of expenditure are often farther subdivided. For example, expenditures for the salaries of 
teachers, administrators, counselors, and "classified" staff may be accounted for separately. 
Furthermore, many systems also distinguish expenditures by "function;" that is, they separate 
expenditures for instruction from those for administration, support services (such as guidance and 
counseling), and operations and maintenance. Capital expenditures are also routinely distinguished 
from operating outlays. 

For most elementary and secondary school districts, budgeting and accounting are limited to 
maintaining fiscal information by these objects of expenditure. With the exception of separate 
budgets for special education and extramural athletics, it is rare to find elementary and secondary 
districts maintaining any kind of budgets by program. Hence, with a few exceptions, most secondary 
districts cannot readily report expenditures for vocational education, nor do their accounting systems 
lend themselves to an easy study of the costs of vocational education. 

At the postsecondary level, many institutions do maintain departmental budgets. At first 
blush, departmental accounting might appear to provide the means for estimating postsecondary 
expenditures for vocational education. This tactic, however, is thwarted by the second problem 
affecting attempts to isolate spending for vocational education: the fuzzy boundaries. 



*For a brief history of VEDS and some of the problems associated with collecting national data on 
vocational education, see E. Gareth Hoachlander, National Data Needs for Vocational Education, 
Berkeley, California: National Center for Research in Vocational Education, October 1989. 



3 See U.D. Department of Education, National Center for Education Statistics, Financial Accounting: 
Classifications and Standard Terminology for Local and State School Systems, Handbook 11, Revised, 
Washington, D.C.: U.S. Government Printing Office, 1973. 
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Can Vocational Education be Clearly Identified? 



Federal law defines vocational education as "organized educational programs offering a 
sequence of courses which are directly related to the preparation of individuals in paid and unpaid 
employment in current or emerging occupations requiring other than a baccalaureate or advanced 
degree/ 4 Hie difficulty with defining vocational education in this fashion is that the same course can 
be taken by students preparing for occupations requiring other than a baccalaureate degree and by 
students preparing for an occupation requiring such a degree. 

For example, accounting classes in community colleges are taken both by students intending 
to transfer to a 4-year college and major in business and by students who will complete their 
postsecondary education with an associate degree or leave before receiving any degree or certificate. 
Similar situations exist in a significant number of other courses offered in business, health, computer 
science, agriculture, and technical education. Consequently, even if departmental budgets exist, 
determining what expenditures might appropriately be classified as vocational education is not a 
straightforward task. 

This kind of ambiguity is also present at the secondary level. Typing is usually offered by 
high school business departments and is taken by large numbers of students who will pursue 
baccalaureate and advanced degrees. The same is true of courses in introductory agriculture, 
computer programing, and electronics. Ironically, in full-time vocational high schools, where 
isolating spending for vocational education might seem the most straightforward, the problem is no 
simpler. For example, at Aviation High School in New York City, about half the graduating class 
will attend 4-year colleges and universities, and many will pursue advanced degrees in aeronautical 
engineering. 

Can the National Assessment determine what vocational education costs in the United States? 
Without a great deal of very expensive data collection, the answer is certainly "no," or at least "not 
with any great precision." Accurately estimating expenditures for vocational education would first 
require some consistent, albeit rather arbitrary, procedures for defining vocational education. Second, 
these procedures would need to be applied to a sufficiently large number of secondary and 
postsecondary institutions to obtain reliable figures. Such a feat could be accomplished only with a 
well-trained cadre of field investigators able to find their way through local accounting systems to 
extract the information needed to establish program costs. Such an operation must entail on-site field 
work; it cannot be done through mail or telephone surveys. 

In short, if accurately calculating the costs of vocational education is possible, it will be an 
expensive proposition. The National Assessment should have serious reservations about the wisdom 
of spending substantial resources on such an effort. We already know that federal spending is a small 
portion of total spending for vocational education, and this fact is not likely to change in the 
foreseeable future. Federal policy, therefore, must focus on a few well-articulated objectives. If 
federal resources are to have a clear impact, they must be targeted and not diffused so widely as to 
have no discernible effect. Moreover, because federal dollars operate at the margins of the 
enterprise, they are more likely to be effective when used as incentives to bring about particular 



♦Carl D. Perkins Act of 1990, Section 521 (41). 
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changes rather than as the basis for broad legislative mandates for rapid, large-scale programmatic 
change. In other words, the policy implications are quite clear, even in the absence of a firm estimate 
of total spending for vocational education. 

Are there any "quick and dirty" alternatives? Perhaps. The Schools and Staffing Survey 
(SAS), conducted by the National Center for Education Statistics (NCES), collects data on teachers 9 
salaries and teaching assignments. Therefore, SAS data can provide national estimates of 
expenditures on teachers* salaries for secondary vocational education, as well as on salaries for 
nonvocational education. The ratio of teachers' salaries for vocational education to teachers' salaries 
for nonvocational education, multiplied by total expenditures for secondary education, may be a 
reasonably good estimate of total secondary expenditures for vocational education. 

It is true that some vocational education courses, especially in trade and industry, health, and 
some technical programs, use supplies and capital equipment more intensively than nonvocational 
courses. Hence, this approach to estimating vocational education expenditures may fall somewhat 
short* However, the error is not likely to be large. The major cost differences per student between 
vocational and nonvocational courses are most likely to be a function of smaller class size in 
vocational courses, and these differences will be accounted for in the SAS data. 

To estimate postsecondary expenditures, a similar approach using the Postsecondary Faculty 
Survey, maintained by NCES, might work. Although it should be reasonably successful in estimating 
expenditures in public, 2-year institutions, the data on proprietary schools are not good, and 
estimating expenditures in this part of the postsecondary enterprise will remain problematic. 

In summary, obtaining information on total expenditures for vocational education directly 
from secondary schools and postsecondary institutions is fraught with problems which are expensive 
and time consuming to overcome. Using national data on salaries of secondary teachers and 
postsecondary faculty may provide the easiest, cheapest means for obtaining a rough but useful 
estimate of total national spending for vocational education. 

WHAT DO FEDERAL DOLLARS BUY? 

The new federal legislation is very explicit about the uses of funds states receive under the 
basic grant. Section 235(a) states that each recipient shall use basic grant monies "to improve 
vocational education programs, with the frill participation of individuals who are members of special 
populations." Furthermore, the act is quite clear about the kinds of activities constituting program 
improvement. These include: 5 

• upgrading curriculum 

• purchasing equipment 

• inservice training 



5 Ibid, Sec. 235 (2) 
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guidance and counseling 



• remedial courses 

• adaptation of equipment 

• tech-prep education 

• supplementary services 

• a special populations coordinator 

• apprenticeship programs 

• economic development programs 

• training adults and students in "all aspects of the industry" 

• comprehensive mentor programs 

• arrangements with private vocational training institutions, employers, 
labor organizations, and joint labor-management programs that can 
provide training at lower cost or training that is not available at public 
institutions 

Short of using federal dollars to maintain routine programs or routinely replace supplies and 
equipment, there is very little on which federal money cannot be spent. 

Tracing how federal money for vocational education is spent should not be very difficult. At 
the state level, good records are maintained on the distribution of federal dollars to local recipients. 
These records will usually be available from the state director for vocational education, although in 
some cases, to obtain the most recent data on distributions it may be necessary to obtain secondary 
and postsecondary allocations separately from the respective state agencies overseeing secondary and 
postsecondary vocational education. It will be easy to determine what secondary school districts and 
postsecondary institutions received Perkins funds. As the money will be targeted under the new 
legislation, sampling from a list of state-identified recipients will probably be more efficient than 
sampling from national listings of secondary and postsecondary agencies. 

When a sample of recipients of federal vocational education funds has been established, 
obtaining more detailed information on what was done with federal dollars should also be relatively 
easy. Local recipients generally maintain good records on how federal dollars are spent. Business 
offices of secondary school districts and postsecondary institutions should have few problems 
supplying information on the kinds of activities for which federal vocational education dollars were 
used. 

Although federal audit trails are usually quite clear, what federal dollars really buy can remain 
ambiguous. This ambiguity results from the suhstitutahility of federal dollars for state and local 
dollars. For example, suppose a local school district desires to purchase new equipment and start a 
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new program. In the absence of federal aid, the district purchases the equipment and forgoes the new 
program. With federal aid, it chooses to use the federal dollars to buy the equipment, because it is 
easier to maintain an audit trail on the equipment purchases, and it uses the state and local money it 
would have spent on equipment to start the new program. Although the books show that the federal 
money was used to purchase equipment, in fact, the federal dollars enabled initiation of the new 
program, because in the absence of federal aid, the equipment would have been purchased and the 
new program foregone. 

Unfortunately, there is no satisfactory way to identify these substitution effects, and the 
National Assessment's efforts to identify what federal dollars buy will be forced to rely simply on 
what the records show. As long as one recognizes that these records may not tell the complete story, 
the resulting picture will be about as accurate as possible. 

ISOLATING THE EFFECTS OF FEDERAL SPENDING FOR 
VOCATIONAL EDUCATION 

In addition to knowing what federal dollars buy, ideally Congress would like to know what 
impact these expenditures have on vocational education. What evidence is there that federal spending 
per se resulted in the improvement of vocational education programs? Unequivocally answering this 
question will probably prove impossible, and the National Assessment should therefore devote its 
efforts to describing changes in the vocational education enterprise as a whole, rather than attempting 
to isolate the impact of federal spending alone. 

There are several reasons for focusing on the larger picture. First, because federal funds do 
represent such a small share of total expenditures for vocational education, Congress needs to know 
about the condition of the entire enterprise if it is to make sensible decisions regarding where limited 
federal resources should be concentrated. Second, even if federal policy objectives are achieved, 
these outcomes may not be a direct result of federal spending. Again, given the relatively small share 
federal dollars represent, it is unlikely that federal objectives that are totally incongruent with state 
and local objectives will be achieved. Federal policy can lead and can provide additional resources to 
hasten changes that states and local agencies are also pursuing. However, it is unlikely that federal 
policy and federal spending can produce proposed changes if those changes are universally resisted by 
the field. 

One should not, however, underestimate the influence of the federal dollars for vocational 
education. While it is true that these resources represent less than 10 percent (maybe even as little as 
5 percent) of total spending for vocational education, they nevertheless represent close to 100 percent 
of the discretionary funds local secondary and postsecondary agencies have to spend on vocational 
education, especially given the increasingly tight fiscal constraints in many states. Hence, federal 
monies are probably the major source of funds for innovations and changes that require new 
resources. To the extent such innovations and changes can be documented, these would seem to be 
legitimately described as "consequences" of federal expenditures for vocational education. 

CONCLUSION 

In summary, the National Assessment should be satisfied with rough estimates of total 
spending for secondary and postsecondary vocational education and should avoid a costly study to 
calculate this total more precisely. No matter how precisely this figure is computed federal dollars 
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as a percentage of the total will represent a small figure. Attempts to describe what federal dollars 
buy for vocational education should not be very difficult, as long as one recognizes that there are 
potential substitution effects that may confound these findings. However, because federal dollars can 
no longer be used for routine program maintenance and because federal dollars probably represent the 
bulk of discretionary money available locally, these substitution effects may be negligible. 
Consequently, what was bought with federal money should represent a fairly accurate picture of the 
impact of federal expenditures for vocational education. 
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Return of the Debate: 
Can Federal Policy Improve Vocational Education for 
Special Populations? 



Lana D» Muraskin* 



For special populations, the Perkins Act of 1990 is both a culmination of, and a departure 
from, previous federal policy. The Act reiterates and underscores congressional concern that federal 
vocational education policy be a means to improve the quality of vocational education for special 
needs groups. At the same time, the Act expands the opportunities for reform of vocational education 
in ways that could divert attention from the needs of disadvantaged and handicapped students. 1 
There is understandable uneasiness among special population groups about the implementation of 
several new or changed provisions. Studying the impact of these provisions on special populations 
will be a major challenge facing the new National Assessment. This brief paper spells out some of 
the issues the assessment may wish to consider. 

SPECIAL POPULATIONS UNDER THE 1984 PERKINS ACT 

The 1984 Perkins Act was the culmination of what was (with the~benefit of hindsight) a two- 
decade long movement to redirect federal vocational policy. The 1963 Vocational Education Act first 
authorized the use of funds for persons with "academic, socioeconomic, or other handicaps that 
prevent them from succeeding in the regular vocational education program" (as well as adults needing 
training). The 1968 Amendments introduced the idea of funds specifically earmarked or "set aside" 
for special populations. These earmarked funds grew considerably over time, and in the 1984 Perkins 
Act, they accounted for over half of the basic grant to states. In addition, the 1984 Act prescribed a 
formula for the within-state distribution of set-asides for disadvantaged and handicapped students, 
added an "equal access" provision, and created a limited service entitlement, identifying certain 
"required" services for disadvantaged and handicapped secondary students enrolled in vocational 
education. 

The 1984 Perkins Act was hailed by groups representing a wide spectrum of special 
population groups. However, it came during a period in which prescriptive categorical programs 
were falling from national favor, and therefore it also seemed to be a throwback to the previous 
decade (if not earlier). Perhaps not surprisingly, some of its special population provisions hardly got 
off the ground. The provision guaranteeing all students equal access to vocational programs and 
services was defined in the regulations to apply only to those programs receiving federal support. 
The means to implement the law's entitlement provisions were never spelled out (the regulations 
simply restated the law) and the scope of these provisions remained murky. A small growth industry 
developed in vocational assessments for disadvantaged and handicapped students, the one entitlement 



*The author is associated with SMB Economic Research, Inc., Washington, D.C. She worked on 
the previous National Assessment of Vocational Education (NAVE). 

^e Perkins Act refers to handicapped students, hence this paper also uses that terminology. 
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the law clearly required. However, many practitioners remained unclear about how to use assessment 
findings to improve students 9 education. 

The Act's provision for dollar set-asides for special populations received the greatest attention. 
Disadvantaged and handicapped students could lay claim to almost a third of the basic grant funds not 
retained at the state level. Further, the law specified that the funds could be used only to pay for up 
to one-half of the "excess costs" of services to students who were disadvantaged or handicapped and 
were enrolled in vocational education. Set-asides were added for displaced homemakers and for 
persons in nontraditional vocational offerings for their sex. These provisions were Congress* attempt 
to target resources to students in need; that is, groups held to be underserved in vocational education 
or to be enrolled in poorer quality programs. 

The story of what happened to the set-aside funds is too long to discuss in detail here. 2 
Suffice it to say that funds for disadvantaged and handicapped students for the most part were 
expended nominally to provide instructional and support services for the intended groups. It was 
almost impossible to know with any assurance whether the funds resulted in more or better vocational 
services or opportunities than would have occurred in their absence. Funds distributed under sex 
equity and single parent set-asides appeared likely to be used for activities that probably would not 
have occurred in their absence. It was almost impossible to account for the uses of funds under the 
adult set-aside at the local level. 

All of these outcomes were overshadowed by two other findings about funds expenditures that 
proved influential with Congress. The first was that the intrastate formula that distributed 
disadvantaged and handicapped set-asides did not appear to increase funds to districts with high 
concentrations of family poverty. The second was the finding that the median grant to a regular 
school district in 1987-88 was only $7,900. The median disadvantaged and handicapped set-aside 
grants were $4,000 and $3,000 respectively. Perhaps more than any other, these findings spurred 
congressional action aimed at new and more stringent targeting of funds. 

SPECIAL POPULATIONS UNDER THE 1990 PERKINS ACT 

As noted previously, some of the important changes enacted in the 1990 Act follow in the 
direction established and maintained since 1963. Reacting to the inability of previous legislation to 
target federal resources effectively on districts and institutions with disadvantaged students, Congress 
has now made the number of disadvantaged and handicapped students the major determinant of the 
allocation of three-fourths of the basic grant funds. Of funds for secondary education, 70 percent are 
to be distributed on the basis of Chapter 1 eligibility, 20 percent on the basis of the number of 
handicapped students with IEP's, and 10 percent according to enrollments. 

At the postsecondary level, funds are based on the institution's percentage of statewide Pell 
Grant and BIA funds recipients who are enrolled in vocational programs. Further, within recipient 
districts, federal funds are to be concentrated in schools with high proportions of disadvantaged and 



*Those readers who wish to explore this issue are directed to GAO testimony by William J. Gainer 
before the House of Representatives Committee on Education and Labor, March 7, 1989, and to Lana 
D. Muraskin, The Implementation of the Perkins Act, Volume II of the final report of the National 
Assessment of Vocational Education (Washington, DC: U.S. Department of Education) 1989. 
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handicapped individuals. Minimum district award levels at the secondary level are set at $15,000, 
and minimum institutional awards at the postsecondary level at $50,000. Districts may form consortia 
to meet the $15,000 minimum grant requirement. 

In addition, the set-asides for single parents/displaced homemakers and persons enrolled in 
nontraditional programs for their sex are transformed into a state-level discretionary grants program 
with all awards to be made competitively. Together, the sex equity and single parent/displaced 
homemaker grant programs account for 10.5 percent of the basic grant funds, similar to the amounts 
set aside for these groups under the 1984 Act. Finally, the 1990 Act has strengthened the equal 
access provision, making it even clearer that it applies to all vocational programs, not just those with 
federal support. 

Although it provides for greater targeting of funds, the 1990 Act also includes a departure 
from previous federal policy for special populations in vocational education. At the same time it has 
redistributed funds in a more focused manner, Congress has become less concerned with tracing 
federal funds through the myriad levels of government to specific individuals. Instead, it has become 
more concerned with using federal resources to spur institutional and district reform in vocational 
education. Within that broad goal, federal funds are intended to help districts and postsecondary 
institutions with concentrations of disadvantaged and handicapped students bring those reforms to 
fruition. 

As a result of this concern for reform in vocational education, the Act also includes changes 
that provide more local discretion (or flexibility) in the uses of federal resources. First, the 1990 Act 
eliminates the set-asides and the excess cost provision. These changes should please every state or 
local official who complained about the paperwork burden of these provisions. However, the changes 
also mean that schools and districts are under no obligation to demonstrate that they provide more 
vocational services to specific disadvantaged and handicapped students with Perkins funds than are 
provided to other students. The Act does contain a comparability provision of sorts. It is enunciated 
in the form of an assurance that state and local funds will be used in the schools of each local 
education agency (LEA) receiving federal funds to "...provide services which, taken as a whole, are 
at least comparable to services being provided in schools in such agency which are not receiving 
federal funds." 

A second change is that states must make the division of funds between secondary and 
postsecondary levels explicit, and must provide a rationale for the split. But in cases where a state 
puts 15 percent or less of its basic grant funds into one of those two sectors, it does not have to use 
the specific "disadvantagement" factors cited earlier to distribute funds to the sector with fewer 
resources* This means that if most of the funds go to postsecondary education, funds that flow to 
secondary education are not subject to the intrastate distribution rules. 

In a third change, there is general language but no specific guidelines on intradistrict funds 
distribution. Districts and postsecondary institutions are urged to put funds into "sites or programs" 
with concentrations of disadvantaged students. They are also urged to concentrate funds in a limited 
number of sites, so that programs are of "sufficient size, scope, and quality to be effective." The law 
provides no minimum school-level grants, however, and it remains to be seen how the regulations 
will define the school/program concentration and "sufficient size, scope, and quality" provisions. 
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Most importantly, districts and postsecondary institutions do not have to use the federal funds 
to support education of disadvantaged or handicapped persons. Once certain distributional 
requirements are met, administrators and instructors may use the local funds for any activity they see 
fit. It should be noted that though the 1984 Act specified in detail which students were eligible for 
support, a district could theoretically concentrate all funds under a set-aside on one eligible student if 
it had a notion to do so. Nonetheless, this change more than any other has many advocates for 
special populations worried that disadvantaged and handicapped students will be neglected. The 1990 
Act clearly tries to shift decisionmaking about the expenditure of federal funds to the district and 
school levels and to give officials at those levels far more flexibility than they had heretofore. Many 
are concerned about what that flexibility will mean in practice. 

If there are "checks and balances" in the new law (i.e., ways of keeping expenditures focused 
on those with special needs), they occur because the fiscal provisions are not meant to stand alone. 
There are potentially sweeping provisions in the new law that are not fiscal but rather seek to make 
vocational education accountable for its expenditures through its results. The greater flexibility in 
funds allocation is balanced by state and local accountability for student performance. 

The 1990 Act calls upon states and localities to create and implement standards and measures 
of performance for vocational education, with special attention to the performance of disadvantaged 
and handicapped students. Standards and measures are to be developed in areas including academic 
and vocational skill competency, school retention and completion, additional training, and job 
placement. Further, the standards and measures system must specifically address the performance of 
disadvantaged and handicapped students, and must provide incentives or adjustments to encourage 
services to special populations and the means for integrating vocational education with IEPs. 
Localities and, ultimately, states are to take action when local performance fails to improve. Thus, 
the flexibility in funds expenditure is to be balanced by a system of accountability for results. 

THE STATUS OF DISADVANTAGED AND 
HANDICAPPED STUDENTS IN VOCATIONAL EDUCATION 

The changes aimed at greater outcome accountability are being enacted at an opportune time. 
Studies conducted for the previous national assessment found changes in vocational enrollments that 
called into question some of the conventional wisdom about the participation of special populations 
and demonstrated the need for greater attention to their opportunities and their performance. 

Studies of course-taking patterns over two decades found that while all students take 
considerable amounts of vocational education in high school, disadvantaged and handicapped students 
take considerably more than other students. In the class of 1987, the average student took about 18 
percent of his or her high school coursework in vocational education. 3 In that same year, the 
average student with an IEP took 27 percent of his or her credits in vocational education. Contrary to 
popular belief, however, most handicapped students were not preparing for work in low-level service 
jobs. 



3 The discussion cited in this section is derived from Becky John Hayward and John G. Wirt, 
Handicapped and Disadvantaged Students: Access to Quality Vocational Education, Volume V of the 
final report of the National Assessment of Vocational Education (Washington, D.C.: U.S. Department 
of Education) 1989. 
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With respect to the relationship between disadvantage and vocational coursetaking, the typical 
student in the class of 1987 who received M mostly AV in high school took less than three credits in 
vocational education, while the typical below-C graduate took almost five credits. 4 Further, the 
vocational coursetaking of mostly-A students fell a full credit between the classes of 1982 and 1987, 
while the credits of below-C students remained virtually unchanged. And while they took more 
vocational education, these lower-performing students took considerably less academic instruction— an 
average of about 13 credits over 4 years for below-C students, compared with about 19 credits for 
mostly-A students in the class of 1987. 

Though they took more vocational education, disadvantaged students were more likely to be 
concentrated in schools with fewer vocational opportunities. 5 In a separate analysis, schools with 
higher concentrations of economically disadvantaged students were less likely than other schools to 
provide opportunities for students to attend specialized vocational facilities such as area schools and 
less likely to offer cooperative education programs that combine instruction with work experience. 
Perhaps most importantly, students in these schools had access to the most limited variety of 
vocational course offerings— about 40 percent fewer choices than schools with the most advantaged 
populations. 

These data show that special populations appeared not to be underserved by vocational 
education, but rather to be major users of secondary vocational services. In fact, the data hint at a 
shift toward a secondary vocational enterprise dominated by special populations, although data from 
additional graduating classes will be needed to confirm that such a trend exists. At the same time, the 
vocational options available to students in schools with high concentrations of disadvantaged students 
remained the most limited. 

These data also confirm that the plan passed by Congress makes intuitive sense. Sufficiently 
large grants targeted to districts and schools with high concentrations of disadvantaged students but 
spent to improve the overall programs in those schools would be a reasonable way to implement the 
federal agenda and address real problems in vocational education. With a system of standards and 
measures that focuses on the performance of special populations in vocational education and can 
trigger action when districts or institutions fail to meet their goals, the 1990 Act changes the terms of 
local and state accountability. Accountability moves from being input-oriented; that is, demonstrating 
that a particular eligible student gets a specific supplementary service (no matter how marginal), to 
being output-oriented; that is, demonstrating the educational and job outcomes of vocational 
education. 

STUDYING THE IMPLEMENTATION AND IMPACT OF THE NEW PROVISIONS 

It is a nice plan, but will it work? There are a variety of issues the new National Assessment 
will need to address do begin to answer that question. The study can examine the ways in which the 
provisions are defined in the regulations and implemented. It can also look at the impact of 
implementation on the distribution of federal support and on the services provided to special 



4 John Tuma et al., Enrollment Trends in Vocational and Academic Education in American Public 
High Schools (Berkeley, Calif.: MRP Associates) 1989. 

5 Hayward and Wirt, op cit. 
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populations. However, given the time frame of the assessment, it will be almost impossible to 
examine the ultimate impact— the educational and employment outcomes for special population 
students. This set of questions will have to be tackled in later years. Some of the items for possible 
inclusion in the National Assessment study design are outlined below. 

Federal Implementation: Regulations and Other Guidance 

In the last Perkins Act, the regulatory process played a major role in determining the course 
of program implementation. Two items noted previously were of particular relevance to special 
populations: the equal access provision of the Act and the various entitlements or required services 
for disadvantaged and handicapped students. Not only were the entitlements simply restated in the 
regulations without explanation of their scope, but also the regulations never clarified whether or not 
districts had to provide services beyond those that could be financed with federal funds. 

In the current legislation, regulatory explanations of several items will be particularly 
important for special populations. The assessment will want to examine how the regulations: 

• Interpret the mechanisms for exemptions to the Pell Grant/BIA funds distribution 
procedure for intrastate distribution of funds to postsecondary institutions (and 
examine the actual waivers granted to states for the use of other distribution 
mechanisms). 

• Interpret the within-district allocation provisions—especially those establishing "site 
and program" priorities and elaborating the comparability provisions. It is particularly 
unclear how the comparability provision should be interpreted at the postsecondary 
level. 

• Elaborate the assurances with respect to required services for handicapped, 
disadvantaged, and LEP students. As mentioned previously, these issues were not 
defined in regulations that accompanied the 1984 Act and now that the required 
services are in the form of an assurance, their status is even murkier. 

• Establish the obligations of postsecondary institutions with regard to the assurance on 
required services. These institutions were explicitly exempted from the service 
requirement in the previous legislation but do not appear to be exempted in the 1990 
Act. 

• Enable states to tell if a program is of "sufficient size, scope, and quality" to warrant 
federal support. Clearly, the interpretation of this provision will be critical to 
ensuring that sufficient funds are directed to reform of vocational education. 
Interpretation may be most critical in those cases where districts form consortia to 
obtain the minimum $15,000 grant award. 

• Describe the methods for funds distribution to area schools and the methods for states 
to determine when these institutions are serving large concentrations of handicapped 
and disadvantaged students. Data from the previous national assessment suggested 
that disadvantaged and handicapped students take more of their vocational education in 
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area schools than other students, but all students take the bulk of their vocational 
education in comprehensive high schools. 

The research to examine these issues is typically analysis of the law and regulations. Given use of the 
negotiated rulemaking process this time, an analysis of that process, its role, and its impact might also 
be warranted. Also, the federal government is charged with providing guidance to states in 
development of performance standards and measures, as well as with administering a variety of 
discretionary programs with great potential to affect disadvantaged and handicapped students. 

State Implementation 

Even before regulations are in place, states will begin to administer the Act. For example, 
they must write state plans, establish committees to guide development of the performance standards 
and measures and begin the development process, and issue guidance to localities on applications for 
support. In a broader sense, they send out signals to localities about how to interpret what they read 
in the Act and regulations and how to implement programs that will pass muster. 

State officials can make or break the reform elements in any legislation, by embracing the 
spirit of the Act or by finding ways to maneuver around it. As a result, it is worth paying particular 
attention to state implementation. In tracking state implementation, special population issues to 
describe and assess include: 

• State rationales for distribution of funds among secondary and postsecondary eligible 
recipients. The last national assessment documented the divisions of funds, but now it 
is important to understand not only how those distributions have changed over time, 
but also why they occur as they do and the implications for improving vocational 
education for special populations. 

• The range of performance standards and indicators and appropriateness of systems that 
are developed for special populations. In addition, to which students do the standards 
and measures apply (i.e., all students or just those deemed "vocational")? 

«» The operation and outcomes of competitions in areas of state discretion with, respect to 
special populations (single parents/displaced homemakers, sex equity, tech-pirep, 
CBO's). In particular, what methods do states establish to ensure funds to places with 
concentrations of disadvantaged students? One finding from the previous national 
assessment was that single-parent/displaced homemaker set-aside funds under the 1984 
Act were not well distributed with respect to community disadvantage. New federal 
rules call for assurance that distribution will go to "individuals with the greatest 
financial need." * 

• Redistribution of unspent allocations. This is also an area in which to pay attention to 
interpretation through regulation as well as to implementation. Are the mechanisms 
used to distribute these funds in keeping with the overall effort to direct resources to 
districts and schools with special populations? 

• State evaluations and initiatives in districts that fail to meet performance standards. 
Although this system will not be in place for a while, it will be important to see what 
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rules states establish for helping recipients who fail to meet the performance goals for 
all students and for special populations. 

• Carrying out all federal requirements on special populations when eligible recipients 
do not report to the agency administering the federal funds. For example, when the 
sole state agency administers secondary education, what guidance is provided to 
postsecondary institutions with respect to the performance standards and measures? 

One way to address most of these issues is by carefully tracking implementation of the 
program in a selected number of states. It would be useful to focus on a limited number of issues and 
to return to state agencies severe times over the course of the implementation period. In carrying out 
this analysis, it is critical to examine implementation of the federal program in the context of ongoing 
vocational and academic education reform in the state. 

Local Implementation 

Studies by the previous national assessment and other sources identified several problems for 
special populations the new legislation has attempted to address. It will be particularly important to 
pay attention to the effects of the new legislation with respect to the following concerns: 

Bifurcation of the federal resources. The divisions in the basic grant under the 1984 Act 
led to a tendency for local officials to view their federal resources as two pots of money— a pot for 
special populations and a pot for program improvement. As a consequence, the two concerns were 
rarely joined. Now that the division between special population funds and program improvement 
funds has been eliminated, are funds being used for improvement of programs with concentrations of 
special populations? 

Narrow service provision for disadvantaged and handicapped students. Because of the 
combination of match, excess cost, nonsupplanting, and required service provisions under the 1984 
Act, special population funds were sometimes spent on services that were marginal to vocational 
instruction and/or of limited value to students (such as assessments that were not used to place 
students in programs, or services based solely on a secondary student's economic disadvantage). Will 
the elimination of the excess cost provision and the greater flexibility the Act provides for the uses of 
funds at the local level mean that funds are more likely to be spent in ways that improve vocational 
instruction for disadvantaged and handicapped students? 

Use of program improvement funds for equipment purchases. To what extent does the 
direction of resources to districts and schools with concentrations of disadvantaged and handicapped 
students also lead to differences in goods and services that are purchased? Does equipment continue 
to play the same prominent role in expenditures as in the past, or are there shifts to other priorities? 

Extremely small grant sizes in regular secondary school districts. Does the increase in grant 
sizes to school districts in combination with the within-district concentration factors help schools and 
districts undertake reforms of broad vocational programs rather than buying small increments of 
supplementary services that lack rationale or focus? 

New issues. In addition to the issues raised in previous research, provisions new to the 1990 
Act raise additional concerns. From the standpoint of ftmds distribution, the most obvious question is 
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whether the new formulas result in greater resources to districts and institutions with the highest 
concentrations of disadvantaged and handicapped students. While studying the distribution of federal 
funds alone says little or nothing about overall expenditures for vocational education, it is clear that 
the Congress, practitioners, and advocates want an answer to this question. The author's 
recommendation is to repeat the survey of school districts and postsecondary institutions carried out 
for the previous assessment. The sampling frame of the previous study will also allow an 
examination of how the new Act may change the distribution of funds to area vocational districts 
(although poverty indicators are not available for those districts from Census data that are usually 
used to analyze fund-distribution information). 

Other funds questions triggered by new provisions include the following: 

• What problems arise when schools that received few or no federal funds in the past 
become major recipients? 

• What problems arise for recipients whose grants are substantially reduced (especially 
in the current climate of fiscal retrenchment)? 

• Can the comparability provision be implemented effectively (since most schools have 
no idea how much they spend on vocational education)? 

• Who participates in development of local performance standards and indicators and 
what measures and standards are adopted? 

• Does any district fail to meet its performance goals? If so, what happens? Does the 
state become involved in joint planning as prescribed by law? 

• How does any of the ambitious reform and assessment agenda operate at the 
postsecondary level, particularly in institutions that are not exclusively vocational, 
such as community colleges? To what programs or students does it extend? 

These questions can be answered by a combination of survey and case studies. 

CONCLUSION 

When field work was conducted for the previous national assessment, vocational educators 
sometimes described the federal program as two programs— "theirs and ours." Many state and local 
officials viewed the expenditure of funds on disadvantaged and handicapped students as the quid pro 
quo for federal program improvement funds (sometimes referred to as "federal equipment money"). 
It would be encouraging to hear that a school used the less encumbered funds of the 1990 Act to link 
academic and vocational instruction in a novel manner or that it created a new program to meet a new 
employment demand in the community. If this Act helps to bridge the gaps between vocational 
expenditures for disadvantaged or handicapped students and those for everyone else, it will have gone 
a long way in improving the opportunities for special populations in training for work. 

The author suggests a simple, intermediate indicator of local success in program 
implementation: Do vocational educators change the ways they describe their expenditures under the 
federal program or their beliefs about what the Act is designed to accomplish? 

85 



85 



Design for the Congressionally Mandated Study of 
the Formula for Distributing Federal Vocational 
Education Funds to the States 



Stephen M. Barro* 



Federal vocational education funds under the Perkins Act are distributed among the states 
according to a statutory federal formula, based primarily on each state's population and per capita 
income. This formula has become something of a historical monument. Over the years, most other 
key provisions of the federal vocational education program have changed drastically. The federal 
government's goals in vocational education have evolved; the intended beneficiaries of federal aid and 
the intended uses of funds have been redefined; and last year, the rules for distributing federal funds 
within states were completely rewritten. But the interstate fund allocation formula today, under the 
1990 Perkins Act, remains essentially the same as it was in 1963, when the Vocational Education Act 
first became law (see note 1).** 

At last, however, Congress may be ready to re-examine the interstate formula as well. A 
separate section of the statutory mandate for a new National Assessment of Vocational 
Education— P. L. 101-392, Section 403(d)— calls for a "study of the distribution of federal vocational 
education funds to the States. " The statute goes into some detail, stipulating that the required study is 
to include 1) an examination of the distributional effects of the existing fund allocation formula and its 
component factors, 2) analyses of the effects of incorporating into the formula additional or 
alternative factors, such as state fiscal capacity and fiscal effort, poverty, and educational 
achievement, and 3) an exploration of fund distribution methods specifically designed to direct funds 
to the economically disadvantaged and other special need groups who are the target populations of the 
Act. This paper reviews the problems and issues that will need to be addressed in this formula study 
and offers a general outline of how to proceed. 

General Fund Allocation Issues 

The issues for this federal fund distribution study can be framed at two levels: first, there are 
generic issues that would arise in evaluating the distribution of any type of federal aid or any federal 
fund allocation formula or process; second, there are more specific issues pertaining to the 
distribution of federal vocational education aid under the Perkins Act. Let us consider the generic 
issues first. 



*The author is President of SMB Economic Research, Inc., Washington, D.C. This is an expanded 
version of a paper he prepared for presentation at the Design Conference for the National Assessment 
of Vocational Education (U.S. Department of Education, Office of Educational Research and 
Improvement), held in Washington, D.C, March 13-15, 1991. All views expressed are solely the 
author's and do not necessarily reflect the positions, policies, or plans of the National Assessment or the 
Department of Education. 

**To avoid interruption of the *extual flow, author's notes are at the end of this paper. 
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The broad questions one would pose in assessing any federal fund method (either actual or 
proposed) include the following: 



1. Does the method distribute funds in a manner consonant with the program's goals? Or, more 
specifically, does it distribute funds in a reasonable relationship to the distribution of the 
needs or problems federal aid is intended to address? 

2. Is the fund distribution method equitable? Does it treat places with similar needs equally, and 
does it take due account of relevant differences in circumstances (educational, demographic, 
and fiscal) among the aid recipients? 

3. Is the method technically sound? For instance, do the variables in fund allocation formulas 
represent what they are supposed to represent, are the data reliable and reasonably timely, and 
are the elements combined in an appropriate 'mathematical form? 

4. Does the distribution method have important side effects? In particular, does it create 
incentives to allocate or use funds or resources in certain ways, and if so, are the incentives 
positively or negatively related to the program's goals? 

These generic questions can be translated into more specific questions about the vocational education 
formula and linked to the specific topics, options, and alternatives mentioned in Section 403(d) of the 
Act. First, however, it seems useful to set the context by explaining how the existing interstate fund 
allocation formula works. 

The Existing Formula 

Under the present formula, federal funds, now amounting to nearly one billion dollars per 
year (FY 1991), are given to states mainly according to two factors— a population factor and an 
income-based adjustment factor (see note 2). 

The population factor is actually a weighted sum of state populations in three age cohorts: 15- 
19, 20-24, and 25-65 (see note 3). It is generally interpreted as a rough indicator of the states' 
relative needs for vocational education services. Other things being equal (and apart from the effects 
of certain constraints, to be discussed later), under the formula, each state's share of funds is 
proportional to its weighted count of persons in the specified age brackets. Obviously, this weighted 
population count is related in only the most general sense to a state's need or demand for vocational 
education or actual level of vocational education services— a point to be pursued further below. 

The formula's income-based adjustment factor is designed so that lower-income states get 
more aid and higher-income states get less aid relative to their populations. This adjustment factor is 
calculated according to the equation 

Income factor = (1 - .5 x PCI/PCIUS), 

where PCI is state per capita personal income and PCIUS is the average per capita personal income in 
the United States. The calculation is also subject to the proviso that no state's income factor may fall 
below 0.4 or exceed 0.6 (see note 4). This factor is applied multiplicatively to the previously 
described population factor, and funds are then distributed among the states in proportion to the 
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resulting mathematical products. The effect of the upper and lower bounds on the income factor is to 
allow aid amounts to vary relative to population by a ratio of no more than 1.5 to one; that is, the 
lowest-income state gets 50 percent more aid relative to population than the highest-income state. 
This income factor is generally understood to be an adjustment for differences in state fiscal capacity. 
States with the least ability to pay, as measured by their residents 9 average per capita income, get 
more aid than states with greater ability to pay. 

Anyone familiar with state school finance systems will recognize the mathematical form of the 
federal vocational education formula immediately, because it resembles closely the state aid allocation 
formulas some states use to compensate for differences in the local property tax base (fiscal capacity) 
among school districts (see note 5). Also, the vocational education formula has important similarities 
to the formulas used to determine the federal share of funding for such programs as Medicaid and Aid 
to Families with Dependent Children (AFDC) (see note 6). In other words, the present vocational 
education formula incorporates a fairly standard method of adjusting for differences in per capita 
income among grantees. 

Aid allocations calculated according to the aforesaid formula are subject to two constraints. 
One is known as the "small state" provision. It guarantees each state no less than 1/2 of 1 percent of 
the total funds available (subject to certain qualifications), even if the state would be entitled to only a 
fraction of that amount based on its population and per capita income (see note 7). The second 
constraint is a "hold harmless" provision, guaranteeing each state no less aid than it received in an 
earlier fiscal year (specifically, FY 1985) (see note 8). 

Specific Issues and Alternatives 

With the foregoing description as background, we can turn to the specific formula design 
issues that need to be addressed and the types of alternatives that need to be examined in connection 
with the vocational education formula. The easiest way to present these issues is to focus, one by 
one, on the different types of factors that either appear in the present formula or would be added by 
various proposals to make the formula more equitable or improve it in other respects. This factor-by- 
factor approach fits in well with the Congressional mandate, which consists in large part of an 
enumeration of factors that, in the view of the drafters of Section 403(d), deserve to be considered. 
Conventionally, the factors used in fond allocation formulas are classified as indicators of need, fiscal 
capacity, fiscal effort, and cost. The following four sections of the paper correspond to these 
categories. A separate section deals with formula constraints and other aspects of formula 
mathematics. 

Indicators of Need 

The basic issues that need to be addressed with respect to the need indicator in the interstate 
fund allocation formula are the following: 

• Does the present formula— specifically, its weighted-population factor— take account 
adequately of interstate variations in needs for the vocational education services to be 
aided under the Act? 
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• In particular, is the formula satisfactory for allocating aid to the economically 
disadvantaged and the other special populations who are designated target groups 
under the Act? 

• If the answer to either of the above is negative, what modified or alternative need 
indicator(s) might improve the match between funding and needs? 

The first two questions almost answer themselves. Even before federal goals in vocational 
education were modified to emphasize services to members of particular special need groups, the 
weighted-population factor was a questionable proxy for needs or demands for vocational education 
services. The assumption implicit in reliance on such a factor— that about the same amount of 
vocational education is needed or wanted per capita in each state—is hard to defend. Consequently, 
there was reason even before the recent changes in the Act to ask whether a better need indicator 
could be developed. Now, basing federal fund allocations on general population variables has become 
even more incongruous because of the heavy emphasis placed on providing vocational education 
services to disadvantaged, handicapped, and other special need students, not only under the 1990 
Perkins Act but also under the 1984 Act. 

Considering also that states are now required under the 1990 Act to distribute federal funds 
internally according to numbers of poor and handicapped pupils served by each local education 
agency (LEA), and even to take concentrations of the disadvantaged into account in allocating to the 
building and program level, it seems inconsistent to ignore the distributions of these same target 
populations in distributing funds among the states (see note 9). The neglect of these special 
populations in the present formula is presumably what led Congress to call for an inquiry into "the 
appropriate distribution mechanism to serve the target population of this Act" (Se.. 403(d)(1)(C)) and 
"methods of targeting funds to individuals who are members of special populations, r nticularly 
individuals who are economically disadvantaged...." (Sec. 403(d)(1)(D)). 

Adding indicators of the prevalence of disadvantaged and other special need pupils to the 
formula is clearly feasible. One approach is ready-made: The method Congress prescribed last year 
for distributing vocational education funds within states could be applied to the interstate distribution 
as well (see note 9). Another possibility is to replicate the approach taken in distributing aid to states 
under the Department of Education's Drug-Free Schools and Mathematics and Science grants 
programs, i.e., allocating a portion of the aid in proportion to each state's allocation of Chapter 1 
grants for the disadvantaged instead of according to a population factor. A third option is to emulate 
the pupil-weighing schemes some states have developed to take account of unequal concentrations of 
poor, handicapped, and other special need pupils in distributing state education aid to local school 
districts (see note 10). Such options will need to be considered in the study. 

Apart from the issue of special populations, it would arguably be desirable to replace the 
present population factor with a variable that relates more directly to vocational education. But here, 
unfortunately, we run into a long-standing barrier that impedes improvement not only of fund 
distribution methods but also of many other aspects of vocational education policy: We have no data 
on the magnitude of the vocational enterprise in the states. No usable state-by-state information is 
now collected on vocational enrollment, vocational coursetaking, or spending on vocational education 
services. Consequently, although the idea of allocating funds according to such measures merits 
serious consideration, little can be done to implement or evaluate it without first making progress on 
the data front. The National Assessment probably lacks the capacity to correct these major data 
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deficiencies, but any steps it could take toward documenting the data gaps and developing methods to 
fill them would be a major contribution. 

Fiscal Capacity Indicators 

The vocational education program is unique among major federal elementary-secondary 
education aid programs in that it distributes aid according to a formula that takes state fiscal capacity 
(represented by per capita income) into account (see note 11). That distinction raises what are surely 
the most basic issues concerning the present fiscal capacity adjustment. Why is it present and is its 
presence justified, considering that Congress has not seen fit to make similar adjustments in allocating 
other education grants? These questions must be addressed at a somewhat philosophical level, as they 
touch on such fundamentals as what the federal interest is, or should be, in school finance equity and, 
insofar as there is such an interest, what specific equity principles should be applied. The point of 
having a fiscal capacity adjustment in the vocational education aid formula is, after all, to offset 
differences in the states' abilities to support vocational education programs on their own. Defining 
specifically what the federal fiscal equalization goal is and how far it extends is logically prerequisite 
to evaluating any particular method of carrying it out. 

Assuming that the appropriateness of pursuing fiscal equity in vocational education is 
affirmed, questions then arise concerning the manner in which, and the degree to which, fiscal 
capacity should be taken into account in distributing federal vocational education funds: 

• Should per capita income be retained as the fiscal capacity indicator, or should it be 
replaced with an alternative measure of fiscal capacity? 

• Should the mathematical form of the adjustment factor be kept as is, or is some other 
adjustment method more appropriate? 

• Should the effect of the fiscal capacity adjustment be strengthened or weakened, and if 
so, to what degree? 

Regarding the choice of a fiscal capacity indicator, specialists in public finance have long 
recognized that per capita income is not a very good measure of fiscal capacity and have come up 
with several alternatives. These include, for example, gross state product (GSP); a measure known as 
Total Taxable Resources (TTR), which is a composite of GSP and income; and the Representative 
Tax System (RTS) and Representative Revenue System (RRS) indices, produced by the Advisory 
Commission on Intergovernmental Relations (ACIR). In addition, regardless of which basic measure 
is selected, the issue arises of whether die fiscal capacity factor in an education aid formula should be 
expressed in per capita terms, as at present, or restated in terms of capacity per elementary-secondary 
pupil. The Congressional drafters of the study mandate seem to be quite aware of and interested in 
these options, as capacity measures are mentioned several times. Note especially the reference in the 
statute to a specific alternative, gross state product per school-aged child. There is no question that 
the study will have to include an assessment of alternative fiscal capacity measures and an analysis of 
the distributional effects of substituting them for the present per capita income factor. 

It will also be important to consider alternatives involving changes in the parameter values or 
in the mathematical form of the adjustment for differences in state fiscal capacity. The most 
important parameter changes are those that would replace the value of 0.5 in the expression 1 - 0.5 x 
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PCI/PCIUS with some lower or higher value. This parameter controls the degree to which the aid 
distribution tilts in favor of states with lower per capita incomes. Raising it would strengthen, and 
lowering it would reduce, the redistributive effect of the formula. Possible changes in formula 
mathematics include replacing the present adjustment factor with a simple inverse income factor or 
replacing the formula PCI/PCIUS with the formula (PCI/PCIUS) squared, as in the AFDC and 
Medicaid formulas (see note 6). The effects of such changes, made individually and in various 
combinations, will need to be determined. 

Fiscal Effort Indicators 

A state's fiscal effort to support a specified program or activity is defined as the ratio of the 
state's own financial support for the activity to the state's fiscal capacity. (The term "state's own 
financial support" refers, in this context, to state-local "own-source" revenue devoted to the activity in 
question, i.e., total state and local expenditure for the activity, less federal aid.) For example, if 
fiscal capacity were measured by per capita personal income, then a state's effort to support 
elementary-secondary education in general would be given by 

per capita own-source revenue for education. 
Effort = per capita personal income 

where per capita "own source" revenue for education is defined as total state-local outlay for 
elementary and secondary education net of federal elementary-secondary education aid. Similarly, 
one could compute fiscal effort specifically to support vocational education by replacing the 
numerator in the foregoing ratio with per capita own-source revenue for vocational education (i.e., 
total state-local vocational education spending less federal vocational education aid). There are as 
many alternative ways of quantifying either type of fiscal effort as there are indicators of fiscal 
capacity. For example, fiscal effort might be measured relative to GSP or relative to the RTS fiscal 
capacity index, as well as relative to personal income. ». 

A fiscal effort factor is supposed to represent the degree to which the state draws on its own 
resources, or the financial "sacrifice" the state makes, to support the program or activity in question. 
The usually cited reasons for wanting to link; federal aid allocations to fiscal effort are to "reward" 
states for spending their own money on activities of federal interest or to create fiscal incentives for 
states to allocate more of their own funds to the designated programs or services. No fiscal effort 
factor is included in the present vocational education formula (nor, incidentally, in the formulas for 
allocating aid under other major federal elementary-secondary grant programs), but the Congressional 
mandate says explicitly that such factors should be considered (see note 12). The issues that need to 
be addressed are the following: 

• Should state fiscal effort be taken into account in allocating federal vocational 
education funds, and if so, should states be rewarded for effort to support education in 
general or vocational education in particular? 

• If effort is to be taken into account in distributing vocational education aid, how 
should fiscal effort be measured, and how should a fiscal effort factor be incorporated 
into the fund distribution formula? 
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Whether fiscal effort should be rewarded is essentially a normative question rather than an 
issue that can be settled by research. What research can do is make clear the implications of 
rewarding effort, and of rewarding effort in various ways. In particular, it will be important in the 
study 1) to evaluate any incentive effects that might be created and 2) to show how the equity of the 
interstate distribution of educational resources would be affected, for better or worse, by inserting 
effort factors into the Perkins Act fund allocation formula. 

The distinction between linking federal aid to effort to fund elementary-secondary education 
generally and linking it to effort to support vocational education specifically is crucial in evaluating 
the reward-for-effort idea. If the purpose of taking effort into account is either to reward states for 
supporting vocational education generously or to elicit more generous support in the future, then the 
effort indicator logically required is one pertaining specifically to vocational education. 
Unfortunately, a serious gap in the vocational education data base— the lack of usable state-level data 
on vocational education spending—makes it infeasible at this time to produce such an indicator. Right 
now, the only kinds of fiscal effort indicators that can be constructed and tried out in allocation 
formulas are the arguably less relevant measures of state-local fiscal effort to support elementary- 
secondary education in general. In other words, the lack of basic financial data on vocational 
education precludes a full inquiry into the fiscal effort issue. In this respect, the study may have to 
fall short of what Congress had in mind. 

Cost Differentials 

A factor not yet mentioned that complicates the task of allocating federal aid appropriately is 
that costs of educational resources vary among the states. In particular, the salaries that must be paid 
to attract teachers and other staff (of given qualifications or quality) vary considerably from one state 
to another. When costs differ, equal dollars do not buy equal resources or services; therefore, an aid 
formula that appears to distribute funds equitably may not distribute real purchasing power equitably 
as well. In principle, it would be possible to compensate for such cost differentials by including a 
cost adjustment factor, or cost index, in federal fund allocation formulas. However, no such factor 
appears in the present vocational education formula. The issues therefore arise: 

• Should allocations under the vocational education formula be adjusted for interstate 
differences in the cost of education? 

• If so, what indicator should be used to represent the relative cost of education in each 
state? 

What makes the cost adjustment problem difficult to deal with is that no satisfactory state- 
level cost-of-education index has yet been developed. In the federal education aid programs funded 
under Chapter 1 of the Educational Consolidation and Improvement Act of 1981 (EGA), the cost 
issue is addressed by incorporating into funding formulas a rough cost proxy based on the level of 
per-pupil spending for elementary-secondary education in each state; however, there are reasons to 
question the conceptual soundness of this solution and to believe that it distorts the interstate 
distributions (see note 13). There are alternative cost proxies, some based on teacher salaries or other 
wage and salary data, that seem to be worth looking into. Clearly, assessments of the available cost 
indicators and the effects of incorporating them into the vocational education formula should be parts 
of the study. 
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Formula Constraints and Other Aspects of Formula Mathematics 

As explained earlier, two constraints are appended to the present vocational education 
formula— one setting a floor of 1/2 of the 1 percent of the available funds under each state's allocation 
and one guaranteeing each state no less aid than in a specified earlier year. The study will have to 
consider whether these provisions are justified and, if so, whether they should be retained in their 
present forms. 

The 1/2 of 1 percent floor, or small-state factor, gives the less populous states substantially 
more aid than they would receive under the basic population-and-income-based formula. Attempts 
have been made to justify this aid floor on cost grounds; for example, by claiming that vocational 
education programs have fixed costs that ail states must incur regardless of size or that there are 
diseconomies of small-scale operation. Such arguments are very thin. If scale is relevant, it is 
probably relevant at the district, school, and program levels rather than at the state level. 
Nevertheless, the effects of the present 1/2 of 1 percent rule and possible variations upon it need to be 
examined. Among other things, it may be appropriate to consider whether some alternative method 
of allowing for genuine scale-related cost differences might be introduced, perhaps in the form of an 
explicit scale factor in the formula. 

Hold-harmless provisions, which protect grantees against reductions in aid below previously 
established levels, are usually justified as devices for cushioning education agencies and programs 
against the N shock N of sharp changes in funding levels. However, this justification applies better to 
fractional hold-harmless provisions (e.g., a provision guaranteeing each state no less than 90 percent 
of the aid it received previously) than to the 100-percent hold-harmless provision now written into the 
law. Among the issues that merit attention in the study are whether there should be any hold- 
harmless rule; whether such a rule, if there is one, should be total or fractional; and how long lasting 
the hold-harmless guarantees should be. 

Apart from the question of constraints, the main issue that has been raised about formula 
mathematics concerns possible changes in the form of the adjustment for state per capita income. 
(This subject has been covered under "fiscal capacity indicators.") In principle, it would also be 
desirable to deal with the more general mathematical alternative of switching from the present lump- 
sum fund allocation method (i.e., allocations independent of state fiscal choices) to a formula 
involving federal matching of state-local outlays. Unfortunately, the lack of data on state-local 
vocational education spending effectively precludes consideration of matching formulas, thereby 
eliminating what would otherwise be a significant class of policy options. 

Study Outline and Methodology 

It seems reasonably clear that to respond to the congressional mandate, the National 
Assessment will have to conduct a study that deals with the existing interstate fund allocation formula 
and its implications and an array of alternative formulas and their distributional effects. The 
assessment of the existing allocation system should have two components: 1) an evaluation of the 
conceptual and technical soundness of the present formula and its constituent factors, and 2) an 
analysis of the present pattern of distribution of federal vocational education dollars among the states. 
Similarly, the assessment of alternatives needs to focus on the rationales for, and conceptual merits 
of, possible formula changes and on how such changes would alter the interstate distribution of 
federal aid. 
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The importance of the conceptual portion of the formula analysis (as opposed to the empirical 
analyses of aid allocations) is often underestimated, but it is only the conceptual analysis that can 
clarify the rationales for, and validity of, current formula factors; identify factors that seem to be 
missing from the formula; and assess the overall logic of formula design. Fortunately, the extant 
body of literature on intergovernmental finance (including specialized literature on such topics as 
measurement of fiscal capacity and cost) provides a ready-made framework for conceptual analysis of 
the Perkins Act formula, making it unnecessary to begin the effort with a blank slate. However, 
some new conceptual work will be appropriate, especially on the question of how the states* relative 
needs for federal vocational education aid might best be defined and measured. 

The empirical portion of the study is likely to revolve around a series of formula simulation 
exercises, in which calculations are made of how allocations to the states would be affected if the 
formula were altered, formula factors changed, or new factors introduced. Such simulations are just 
as important for studying the existing formula as for examining alternatives. For example, the 
obvious way to determine the effects of the present 0.4 to 0.6 bounds on the formula's per capita 
income factor is to compare allocations under the current formula against those produced by an 
otherwise identical formula with the bounds removed. A large number of these simulations will be 
needed to look at all the alternatives mentioned in the congressional mandate as well as the numerous 
others that will undoubtedly surface in the course of the study. 

Various statistical tools will need to be used to characterize and compare the current and 
alternative distributions. Several types of standard descriptive statistics (univariate and bivariate) will 
be required, including measures of interstate disparities in aid , differences by region and by other 
groupings of states, and measures of the relationships of aid to such state attributes as population, 
poverty, and minority percentage (correlations, rates of change, elasticities, etc.). More specialized 
statistical indicators will be needed to describe differences between distributions; for example, the 
percentages of aid shifted by a given formula change, the degree to which effects are concentrated on 
particular states, and the pattern of state gains and losses. There is also likely to be a role for 
multivariate analysis (multiple regression and analysis of variance) to determine how aid allocations 
vary in relation to particular state attributes when other attributes are held constant (e.g., how aid 
varies with state fiscal capacity, controlling for state size). But in general, no special or elaborate 
methodology seems to be required for this study; standard, well-established statistical methods should 
suffice. 

Notes on Related Issues and Analyses 

Although one can think of the evaluation of the interstate funding formula as a self-contained 
study, some of the state-level fund allocation issues relate closely to other topics that could be covered 
by the new National Assessment of Vocational Education. The following are brief comments on other 
inquiries that could usefully be coordinated with work on the interstate funding formula. 

The Intrastate Allocation of Federal Aid. The mandate for a fund distribution study in 
Section 403(d) of the 199C Act refers only to the interstate distribution of aid and not to allocations of 
federal funds to local recipients within each state. Nevertheless, it seems desirable to append to the 
interstate formula study at least a conceptual analysis of the intrastate fund distribution mechanism. 
Analyses of both the interstate and intrastate fund allocation methods are needed to produce a 
complete picture of how federal funds are distributed to the ultimate recipients and, more particularly, 
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to assess how well aid is being focused on special need populations. Moreover, similar issues of 
formula design arise, similar concepts and criteria are applicable, and similar alternatives are relevant 
at the two levels. Because the new intrastate allocation rules have yet to be implemented, an 
empirical analysis of within-state distributions cannot yet be undertaken. (It also is not at all clear 
that adequate data would be available to support such research). However, the intrastate formulas 
themselves can be assessed now, and combining that assessment with the state-level analysis could add 
significantly to the value of the whole study. 

Participation in Vocational Education. The present complete lack of state-level data on 
participation in vocational education hampers the formula study in two ways. First, as already 
explained, it prevents analyses of the effects of replacing the present population factor by some 
measure of the actual amount of vocational education service provided by each state. Second, it 
makes it impossible to determine how well aid allocations under any formula match the interstate 
distributions of vocational pupils or services. 

Progress in developing such dat* would open up a number of new options and analytical 
opportunities. However, experience has shown that what appears to be the most direct method of 
measuring participation— counting nvmbers of "vocational students"— is a conceptual dead end. 
Nearly all students qualify as "vocational" in the sense that nearly all enroll in one or more vocational 
courses. A more sensible approach, as shown by the previous National Assessment of Vocational 
Education, is to quantify vocational coursetaking. It is probably too much to expect that state-level 
measures of vocational coursetaking will be developed by the new National Assessment, but any 
progress in that direction would be helpful. Without such measures, we will never be able to 
establish convincingly how well resources are being distributed in relation to needs. 

The Cost and Financing of Vocational Education. Our present ignorance of how much 
states and local agencies spend on their vocational education programs is a serious obstacle to policy 
analysis in vocational education, including analysis of alternative fimd-aliocation formulas. Lacking 
such information, we cannot consider what would otherwise be major classes of formula options, such 
as allocating federal aid on a matching basis or linking allocations to state fiscal effort to support 
vocational programs. The same data gap also rules out such evaluative procedures as comparing 
federal shares of different states' total vocational education funding or determining how changes in 
federal aid would affect the distributions of total support (not only federal) for vocational programs. 
Developing the now-missing cost and financial data would be a major undertaking, but the results 
would be valuable for many purposes beyond developing better fund allocation methods. It is not 
clear what the National Assessment will be able to undertake along these lines, but even some initial 
steps, such as laying out the design for a cost and financial reporting system, would be an important 
contribution. 
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To be precise, the basic fund-allocation formula, according to which aid is calculated as a 
certain mathematical function of state population factors and state per capita income, has not 
changed since 1963, but changes have been made in certain constraints appended to the basic 
formula. 

The appropriation for the main program of federal vocational education aid to states, Basic 
State Grants for Vocational Education, is $848.4 million for FY 1991. However, 
appropriations for grants to support consumer and homemaking education, programs operated 
by community-based organizations, and a newly created category of tech-prep programs, 
which use essentially the same fund allocation formula as the Basic Grants program, bring the 
total up to about $959 million. 

The formula stipulates that 50 percent, 20 percent, and 15 percent of the available funds shall 
be distributed in proportion to populations in the 15-19, 20-24, and 25-65 age strata, 
respectively (modified by the per capita income factor), and that the remaining 15 percent 
shall be allocated in proportion to the resulting totals. This is equivalent to assigning weights 
of approximately 58.8 percent, 23.5 percent, and 17.6 percent, respectively, to the three 
population categories. 

The statute specifies that the per capita income factors to be used in these calculations are 
averages of state per capita personal incomes in the 3 most recent years for which data are 
available. 

For examples of such programs, see R. Salmon et al., Public School Finance Programs of the 
United States and Canada, 1986-1987, American Education Finance Association, 1988. 

The adjustments for state per capita income in the AFDC and Medicaid formulas are based on 
the same ratio, PCI/PCIUS, as is used in the vocational education formula. However, there 
are two important differences in formula structure. In the AFDC and Medicaid cases, 1) the 
variable that actually enters into the formula is the square of the ratio PCI/PCIUS rather than 
the ratio itself, and 2) the per capita income factor is used to determine the federal share of 
program funding rather than the absolute dollar amount of federal aid. These variations are 
potentially of interest as alternatives to the present vocational education formula. 

Under the 1990 Act, the qualifications are that no state may receive, by virtue of the 1/2 of 1 
percent rule, more than the lesser of 1) 150 percent of what is received in the preceding fiscal 
year and 2) no more than 150 percent of the national average per pupil payment in the 
previous year (based on pupils counted for formula allocation purposes). The latter restriction 
was not previously included in the law. 

The hold-harmless rule has had minor effects because the total funds available for distribution 
to the states have increased from year to year. As of FY 1989, only five states gained from 
the rule, of which only two, Massachusetts and New York, gained significantly (S. M. Barro, 
The Distribution of Federal Elementary-Secondary Education Grants Among the States, SMB 
Economic Research, Inc., Washington, D.C., forthcoming in 1991). 
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9. According to Section 231 of the 1990 Act, states are required to distribute 70 percent of the 
funds available for secondary-level vocational education programs to local education agencies 
(LEAs) in proportion to the LEAs' allocations of Chapter 1 grants for the disadvantaged 
(which are based primarily on the incidence of poverty in each LEA) and another 20 percent 
in proportion to the number of handicapped pupils in each LEA. Local aid recipients are then 
required (under Section 235) to "give priority for assistance... to sites or programs that serve 
the highest concentrations of individuals who are members of special populations." 

10. Under state pupil-weighting schemes, state aid to local school districts is allocated according 
to numbers of weighted pupils rather than actual numbers of pupils in attendance in each 
district. Typically, regular pupils are assigned a weight of 1.0; disadvantaged pupils are 
assigned a higher weight, such as 1.25; and various categories of handicapped pupils are 
assigned weights commensurate with the relative costs of their special education programs. 
For examples of such weighting systems, see R. Salmon et al., Public School Finance 
Programs of the United States and Canada, 1986-1987, op. cit. 

11. A per capita income factor of the same mathematical form also appears in the interstate fund 
allocation formula of the Department of Education's Vocational Rehabilitation program. 

12. The congressional mandate uses the term "tax effort" rather than "fiscal effort." The former 
is a somewhat narrower concept in that it takes into account only the states* efforts to raise 
revenue through taxation, whereas "fiscal effort" also reflects the states' efforts to raise 
revenue from user fees and other nontax sources. 

13. The cost index in the Chapter 1 formulas is basically an index of per-pupil expenditure for 
elementary and secondary education, but values of the index are limited to no less than 80 
percent and no more than 120 percent of the U.S. average. The main shortcoming of such an 
index is that it confounds interstate differences in fiscal capacity and/or fiscal effort with 
differences in the cost of education. This confounding results in systematic over-estimation of 
education costs in high-spending states and systematic under-estimation of costs in low- 
spending states. For details see S. M. Barro, The Distribution of Federal Elementary- 
Secondary Education Grants Among the States, op. cit. 
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1990 Perkins Act Funding Issues 
at the Postsecondary Level 



Rosemary Sullivan Zins* 



INTRODUCTION 

Assessing the effectiveness of Perkins programing at the postsecondary level necessitates an 
understanding of the missions and goals of community colleges. The community colleges came into 
existence by virtue of being open-access institutions designed to serve educationally and economically 
disadvantaged students and nontraditional, adult populations. 

As open-access institutions, community colleges have had to retain strict outcome standards in 
order to preserve the integrity of the associate degree and to validate their role as transfer institutions. 
Simultaneously, they have been charged with serving a broad range of students who enroll with 
varying levels of competencies. 

Since community colleges have historically bridged the gap between the educational 
competencies students need to graduate from secondary schools and the educational competencies 
needed to gain admission into 4-year baccalaureate degree programs, any type of performance 
standards used to measure the accountability of community colleges' effectiveness must take into 
consideration a "value-added" approach. A value-added approach makes similar sense when trying to 
assess the effectiveness of the new Perkins Act, especially when the primary thrust of the new Act is 
to serve special populations such as individuals with disabilities, educationally and economically 
disadvantaged students, the incarcerated, women, minorities, immigrants, and nontraditional students. 

In the traditional high school market, community colleges are best noted for their experience 
in serving academically disadvantaged, economically disadvantaged, and underprepared students. 
According to the National Center for Education Statistics* study "High School and Beyond," high 
school seniors who entered 2-year institutions had lower records of academic achievement than those 
at 4-year institutions. Additionally, entering freshmen at 2-year colleges were more likely to come 
from low-income families than their 4-year counterparts. (Carter 1990, 2-3) Community colleges 
have not only been successful in recruiting these populations, but have also proudly established their 
ability to serve and retain these populations. 

The community colleges have identified the nontraditional student (i.e., the student who does 
not directly enter a postsecondary educational institution upon graduation from high school) as a 
primary target market. As such, community college students are generally not "youths" but rather 
adults who are seeking to acquire occupational and job-related skills to improve their employability or 
promotability within the workplace. Community college students are typically about 28 years of age 
and attend school on a part-time basis in order to balance their competing personal, family, and 



♦The author is Director of Development, Community College of Rhode Island. 
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employment demands. At present, the majority of 2-year college students (68 percent) attend part- 
time. 

When developing the National Assessment and designing performance and outcome measures, 
it is important to remember that adult students as well as secondary school youth are being served by 
Perkins-funded programs. It is also crucial to remember that these adult students have elected to 
return to school to continue their educations despite the obstacles that they must overcome, such as 
childcare arrangements, transportation, financial concerns, and the competing time demands of family 
and work. Too often, the Perkins Act is erroneously interpreted as being exclusively for the benefit 
of secondary school youth who are mandated by law to attend school and who attend school on a full- 
time basis. 

Because 2-year college students enroll for varied reasons, their educational goals must be 
considered when outcome measures are being selected for the National Assessment. According to the 
American Association of Community and Junior Colleges, SO percent of 2-year college students stated 
that their primary reason for enrolling was to acquire job-related skills; 36 percent enrolled in 
preparation for transfer to a 4-year college or university; IS percent wanted to fulfill a personal 
interest; and 4 percent said they wanted to increase basic English, reading, or math skills. (Carter, 4) 

Although graduation data may be valid for the 4-year institutions as well as for secondary 
schools, it is inappropriate as the sole measurement for assessing the effectiveness of community 
colleges. Reasons for enrollment must be factored into any such measurement. Since community 
colleges have many students who take courses simply to upgrade their job skills or retrain for a 
different career, the attainment of the associate degree is not necessarily the planned outcome, nor the 
desired goal. 

Retraining and upgrading the work force has become a critical factor in maintaining this 
country's ability to remain technologically and economically competitive in a global marketplace. 
The flexibility community colleges allow their students in registering for one course and/or clusters of 
courses without matriculating in a degree program, and the community colleges' ability to provide 
customized training and flexible scheduling, directly respond to this changing labor market and to 
societal trends. 

DEMOGRAPHICS AND THE DEMANDS OF THE WORKPLACE 

The "graying of America" is underway as the baby boom generation ages and the overall 
U.S. birth rate declines. This is resulting in a reduced pool of young adults and a scarce labor pool. 
Since 84 percent of the year 2000 work force will be adults who are already employed, business and 
industry are focusing on retraining and upgrading the skills of the existing work force. (Johnson and 
Packer 1987, 16). The current skilled worker faces four to five job changes and a dozen skill 
upgrades over the course of his/her career. 

The U.S. Department of Labor projects that 16 of the 20 fastest growing job occupations will 
require some type of training beyond high school at a time when the educational community is having 
trouble simply keeping students in school long enough to obtain their high school diplomas. (Parnell 
1987, 37). 
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"The ominous news within our school systems seems relentless: the declining test scores, 
growing numbers of high school students unable to perform simple arithmetic, poor showings 
of American youth in international comparisons of academic ability, a national dropout rate of 
roughly 25 percent, and forecasts of relatively high demand for workers with technical skills/ 
(Office of Technology Assessment 1989, 10) 

Consider these facts: 

• Of the 40 million elementary and secondary school students in the country today, 11 
million will not finish high school. Two-thirds of the dropouts will come from the 
unfocused general education population. 

• Of those who do graduate, a large percentage will require remediation in reading, 
writing, and mathematics skills when they enter college. 

• Over 42 percent of the high school students in this country are enrolled in general 
education programs. Thirty-six percent are enrolled in college prep, and 19 percent 
are in vocational programs. (Parnell 1989) 

The racial composition in our country is also rapidly changing, with whites a diminishing 
percentage of our youth population. 

• By the year 2000, more than one-third of the nation's population will be members of 
minority groups (Black, Hispanic, Native American, and Asian). Today, while the 
U.S. population of 238 million includes 27 million blacks and 15 million Hispanics, 
by the year 2020, the total U.S. population of 265 million people will include 44 
million black citizens and 47 million Hispanic citizens, possibly more Hispanics if 
their immigration rates increase while their relatively high birth rate holds steady. 

• The minority population in the year 2000 will be much younger, in median age, than 
the white majority population. The Hispanic population will increase four times as 
fast as the white population, becoming the largest minority group by the year 2000 
and having the lowest median age of any minority group. 

• Between now and year 2000, more than half of all new workers hired in the United 
States will be minorities, nearly three times the current rate. Over the next 10 years, 
only 15 percent of the work force entrants will be native-born white males. 

• By the year 2000, the majority of the nation's public school enrollment will consist of 
members of minority populations. 

• Of the new job entrants, the U.S. Department of Labor estimates that most will come 
from groups that traditionally lack vocational education opportunity— women, 
minorities, immigrants, the poor, and the disabled. (New England Board of Higher 
Education 1989, 2) 

The primary concern of more than 80 percent of employers is finding workers with a good 
work ethic and appropriate social behavior: "reliable/ "a good work attitude," "a pleasant 
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appearance/ "a good personality". (National Center on Education and the Economy 1990, 3) A 
portion of these concerns can be attributed to the cultural diversity surfacing in our society and work 
environments. The problems associated with trying to integrate immigrants, minorities, the 
disadvantaged and other previously unemployed or underemployed populations into the work force to 
respond to the increasingly scarce labor supply are compounded by the heightening concerns of 
declining productivity in the workplace. 

Employers are also expressing the need for a different type of worker. As our society 
evolves from a manufacturing economy to a service and technological economy, employers need 
workers who are bright, adaptive, and creative. They want and need "learners" who have good math, 
science, and communications skills. Workers are no longer simply asked to follow directions, but are 
often required to demonstrate problem-solving skills in order to complete their job assignments. 
America's productivity growth slowdown is becoming an alarming issue, especially since we can no 
longer increase productivity simply by adding more workers to the work force. TTie critical issue will 
be "working smarter," not simply hiring more employees or motivating existing employees to work 
harder or longer. Surveys of labor market needs indicate the following: 

• Most jobs will require workers who have solid science, math, and communication 
skills, but fewer than one in four new employees will be able to function at the 
needed level. 

• The types of skills necessary to implement the new computer-based technologies 
require workers to develop broader skills in problemsolving and decisionmaking. 

• Employers are increasingly dissatisfied with the entry-level skills and work attitudes 
of newly hired workers. Two-thirds of all employers consulted in a national survey 
assessed their current pool of entry-level applicants as being insufficiently prepared. 
(Council of Chief State School Officers 1988) 

THE EDUCATION SYSTEM RESPONDS: THE TECH-PREP PROGRAM 

America's quest to develop a world-class work force that can be technologically competitive 
in the global marketplace is going to require immense cooperation and communication between the 
educational community and the economic development arms of business, industry, and government. 
Perkins funds should be used to encourage working partnerships among these entities. Educational 
institutions, both secondary and postsecondary, cannot and should not operate in isolation from the 
employment sector. They must assume more aggressive roles as economic development partners 
within their regions. Coordination between secondary and postsecondary institutions is also essential 
if we are to produce and develop a well-trained work force. 

The tech-prep model offers an alternative grade 11 and 12 curriculum in math, science, and 
communication to better prepare students for the growing market of technical careers. It exemplifies 
integration of academic and vocational education and coordination among secondary education, 
postsecondary education, and the employment sector. The tech-prep model is also one of the best 
examples of how to implement the new Act in accordance with its three major thrusts: 

1. To foster integration between academic and vocational education. 



102 

101 



2. To encourage coordination among postsecondary and secondary educational 
institutions. 



3. To promote increased accessibility and retention of special populations (i.e., 

academically and economically disadvantaged students, minorities, disabled students, 
and nontraditional students). 

Tech-prep's applied curriculum works well for disadvantaged and less motivated students, as 
it provides them with a sequential, goal-directed course of study. At worst, it is a strategy to keep 
students in high school and promote basic skills competencies. At best, it is a strategy to encourage 
students to pursue the option of a technical career by continuing their educations at postsecondary 2- 
year institutions. 

The 2+2 tech-prep initiative, as described by Dale Parnell in his book The Neglected 
Majority, illustrates the overall goals of the Perkins Act, not just the tech-prep component. The 
National Assessment needs to identify models similar to tech-prep for implementation of the new 
Perkins Act. Many vocational educators are still struggling with the terminology and implications of 
the new Act. It would be a travesty if vocational and adult educators were allowed to continue on a 
"business as usual** basis because of the problems in translating the terminology into action and 
because of the lack of examples of successfully integrated programs. 

Assessing Perkins programs under the tech-prep initiative should initially focus on the depth 
and breadth of curricular reform. Factors that need to be analyzed include the degree of curricular 
revision, the level of articulation between secondary and postsecondary faculty, the coordination of 
student support services, and the labor market linkages. 

The National Assessment must recognize that tech-prep programs are still in the evolutionary 
stages, preliminary outcome data on tech-prep students may be misleading as students may not have 
had the opportunity to participate in or complete the entire series of applied math, science, and 
communications courses. These applied courses are still in the process of being introduced and field 
tested. Since tech-prep students are still entering postsecondary programs with varying levels of skills 
and backgrounds, the "value-added" approach makes sense as it does not penalize efforts for 
developing new programs and improving existing programs. 

The success of tech-prep programs will have to be determined by longitudinal studies to 
determine its true impact within the secondary schools, the postsecondary institutions, and the 
workplace. Tech-prep is not a quick fix but rather a long-term systems approach to changing the 
skills of our future work force to meet projected labor market needs. Initially, assessment data on the 
tech-prep program should try to focus on measuring the reactions of students, teachers, and employers 
to the programs through self-report surveys and case studies. 



FUNDING ISSUES 



There are three major issues with respect to Perkins funding at the postsecondary level: 



1. 
2. 
3. 



Getting the dollars—the external politics 
Spending the dollars— the internal politics 
Keeping the dollars— the balancing act 
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Getting the Dollars 

The external politics of "getting the dollars" centers around the postsecondary-secondary split. 
The formula split was a major issue during the reauthorization of the Act, as it politically and 
geographically pitted region against region. It was a classic confrontation of East meets West, as 
most of the eastern states, particularly the New England states, allocated an average of 35 percent or 
less of the Perkins monies to postsecondary institutions. Indeed, some states east of the Mississippi, 
such as Connecticut, Ohio, and Indiana, gave no Perkins dollars to postsecondary institutions. The 
opposite was true in states west of the Mississippi where only one state, Missouri, granted less than 
35 percent of the Perkins monies to postsecondary institutions. (National Assessment of Vocational 
Education Survey 1989) The postsecondary-secondary disagreement was resolved at the national level 
by deciding that the percentage split should be left to the discretion of each state. 

The postsecondary-secondary argument over Perkins monies is an outgrowth of the 
demographic trends in the general population, the declines in secondary school enrollments, and labor 
market trends. The issue of the postsecondary-secondary split fragments the educational community, 
both locally and nationally, because it perpetuates turf issues at precisely the time when educational 
unity is vital. The leaders of postsecondary and secondary institutions are arguing against each other 
over funding instead of working together to improve the educational system's responsiveness to 
changing work force trends and pressures. This problem is especially acute for an Act that is trying 
to promote increased coordination between secondary and postsecondary institutions and is promoting 
the 2+2 tech-prep initiative. 

The external politics of "getting the dollars" involves not only the "how much" issue, but also 
the issue of how the funds flow to the applicants, specifically the allocation vs. the RFP (request for 
proposal) process. Although funds were distributed using both mechanisms under the old Perkins 
Act, the allocations were restricted to the programmatic priorities that were identified by the funding 
source. The applicants merely reacted to the stated priorities by designing distinct and separate 
programs if they wanted to receive the allocated monies. 

Under the new Act, the allocations are restricted to specific applicants rather than specific 
programs. The new Act authorized the applicants to identify their own priorities for spending the 
Perkins dollars and encourages them to develop comprehensive and integrated programs. 
Additionally, the applicants are afforded much greater responsibility, flexibility, and latitude in how 
they spend the allocated monies, as they are provided with a broad range of acceptable and allowable 
program parameters. Since the new Act allows the applicants to assume a proactive rather than a 
reactive role in the design and management of its programs, applicants are provided with a much 
greater incentive for seeking Perkins funding and are given both the responsibility and the necessary 
authority to ensure their own successes and to control their own futures. 

The new Act also has simplified the paperwork for the allocation process. Only one 
comprehensive needs assessment must be completed by the applicant, while under the old Act, a 
separate needs assessment had to be developed and submitted for each proposed program. Under the 
new Act, the allocation process not only requires long-term planning, but also promotes 
comprehensive coordination and communication. 
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Allocations increase the probability that institutions will aggressively seek to coordinate 
Perkins funds with supplemental funds from other sources such as the Job Training Partnership Act 
(JTPA), U.S. Department of Labor; JOBS (Welfare Reform Act), U.S. Department of Health and 
Human Services; or TRIO, U.S. Department of Education-Title IV in order to take advantage of 
economies of scale in serving special populations. These coordinated efforts will minimize the 
overhead costs needed to conduct grant programs, will funnel more funding to direct service delivery, 
and will simultaneously improve the quality and the cost-effectiveness of programs. The allocation 
process will significantly assist the new Perkins Act in its goal of funding a limited number of 
programs which efficiently and effectively serve large numbers of students in a well-coordinated 
delivery system. 

Allocations promote higher quality programs as institutions begin to integrate the grant-funded 
projects into their core of operations instead of characterizing them as free-standing, temporary 
projects. The allocations encourage the institution to invest in the grant programs because they now 
have the flexibility to prioritize programs in accordance with their own established missions. As 
such, the grant-funded programs become much more readily accepted as "essential" rather than 
"expendable," services and have a much better chance of eventually becoming self-sustaining. 

Allocations are a much more effective vehicle for solidifying institutional support than 
requiring matching contributions which often are simply inflated estimates of in-kind contributions 
rather than actual cash matches. 

Under the new Act, competitive RFPs will still be granted but will be largely restricted to the 
set-aside funding for consumer education, community-based organizations, tech-prep, and sex equity 
programs for single parents, displaced homemakers, and single pregnant women. 

Currently, the set-asides are determined at the national level as a percentage of the state's 
basic grant allowance. In the future, it would be preferable to grant the states the same type of 
discretion that was granted in the postsecondary-secondary split. The states' decision-making 
processes could be mandated to reflect demographic trends and labor market demands to ensure 
equitable distributions of this funding. 

Sex equity and special populations 

One of the inherent dangers in promoting specific RFPs for any priority is that institutions 
tend to believe that if they develop grant proposals to address these issues, they do not have to do 
anything else. They also tend to limit funding for those particular programs to the "soft" money that 
is available under the RFP. The National Assessment should ascertain whether or not institutions are 
spending any other monies, either "hard" or "soft," on these priorities. The answers to the questions 
"why?" and "why not?" should be based upon the supporting evidence of demographic and labor 
market trends in the geographic areas being served. 

For example, most regions/institutions will probably choose to limit funding for their tech- 
prep initiatives to the funding available under the tech-prep RFP process. The same hold true for sex 
equity programs. It would be extremely informative if the National Assessment could identify the 
degree to which regions or institutions are using their allocated monies to supplement the RFP 
initiatives. Many institutions probably do not even realize that it is perfectly acceptable to use their 
allocated monies to supplement these efforts. 
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At the postsecondary level, the sex equity set-asides may actually overcompensate, as 
community colleges not only have the tradition of enrolling nontraditional and special populations but 
also have developed a solid reputation for successfully retaining these populations. In terms of 
quantity, women comprise 57 percent of enrollments in the 2-year sector. (Carter, 2) Many of these 
adult students are single parents or displaced homemakers who are functioning as heads of households 
and have returned to school to increase their earning potential to better support themselves and their 
families. 

In terms of quality, most community colleges have been in the forefront of postsecondary 
institutions in responding to the needs of adult learners and nontraditional populations. Equity issues 
may differ substantially at postsecondary and secondary institutions due to the maturity levels of the 
populations being served. Also, adults elect to attend postsecondary programs; they are not mandated 
to attend by law. This disparity should be considered when the National Assessment is being 
developed. 

The emphasis on using set-asides to promote sex equity and special populations issues at the 
secondary level, particularly within secondary vocational schools, may be more justified, as these 
schools have traditionally been male dominated due to the societal and cultural expectations regarding 
vocational education. As the country shifts into more technical education, the sex equity issues 
should eventually resolve themselves as the term technicians 1 * generally does not connote gender- 
specific or gender-dominated occupations. Understandably, the Perkins Act has chosen to attack the 
issue of sex equity by establishing die set-asides. 

However, in an effort to promote increased sex equity under the new Act, the lawmakers have 
gravitated toward adopting quota systems which defy the basic operating principles of affirmative 
action policy. Under the new Act, institutions are required to implement immediate as well as longer- 
term strategies for all programs which have less than a 25 percent enrollment of one sex or the other. 
Quota systems are dangerous because they establish an ideal number which is then interpreted as a 
ceiling for enrollment. 

Measurement of efforts to promote sex equity and involvement of special populations in 
Perkins programs should not focus solely on enrollment statistics, but should consider the degree and 
diversity of retention strategies. Most importantly, institutions which are expending enormous efforts 
to serve special populations should not be penalized because of their failure to attain a specified quota, 
but should be encouraged in the progress they have made. A value-added evaluation approach would 
accomplish this objective. 

Both secondary and postsecondary institutions can and should be assessed as to their 
recruitment and retention strategies for serving special populations. Baseline efforts could include 
measurement of the following: 

• extent to which outreach is conducted with community-based organizations and 
referral agencies 

• staff participation on community boards and councils 

• use of facilities for multicultural events and special populations events 
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staff development activities 



evidence of affirmative action and Equal Employment Opportunity 



availability of adapted equipment for disabled students 



modifications to the physical plant to promote access to individuals with disabilities 



operation of student clubs for special populations 



availability of support services for special populations (e.g., financial aid, English as 
a Second Language courses, career counseling, and tutoring services) 



extent to which publications promote and encourage multicultural, non-sex-biased 
awareness 



extent to which supplemental grant funds are sought to serve special populations (e.g 
JTPA, JOBS, and TRIO funds) 



Since the new Perkins Act places more emphasis on serving special populations, data 
collection, design, and assessment should be coordinated with other federal programs serving the 
same or similar populations, such as the JTPA, TRIO Programs, and the JOBS Act. 

One of the greatest problems special populations face is trying to access the system, be it the 
educational system, the job training system, or the human service delivery system. A tremendous 
amount of time and money is wasted because of the lack of interagency coordination and the 
incompatibility of the separate data reporting systems. A National Assessment of the Perkins Act 
could address this issue and advocate a mutually shared educational and training data base that could 
track clients/students. This same data base could also be used by employers to assess the 
employability and skills level of potentials job applicants. 

Spending the Dollars 

The second postsecondary funding issue involves the internal politics of "spending the dollars" 
within the institution. Internally, a balance must be achieved among the for-credit programs, the 
noncredit programs, and student services. We must look at who makes the decisions and why they 
are made. Particular emphasis needs to be placed on the "tail wagging the dog" syndrome if all the 
funds continue to be targeted to noncredit skill-training programs and if the "soft money" priorities 
substantially differ from the "hard money" priorities of the institution. 

At postsecondary institutions, specifically community colleges, the noncredit programs often 
function as feeder programs to the degree programs. The noncredit, short-term skills training 
programs provide a nonthreatening atmosphere for adults and/or at-risk special populations who have 
been away from the formal education system. Completion of these programs often gives students the 
incentive and the confidence to attempt enrollment in credit-bearing courses. Most importantly, the 
noncredit programs promote the concept of life-long learning which is essential to surviving 
successfully in today's job market. The likelihood of retraining during the course of one f s career is 
an accepted fact in today's changing world. 
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Conducting and evaluating programs within the constraint of 1-year funding cycles is a critical 
issue for postsecondary institutions trying to promote involvement and retention of special 
populations. For example, individuals with disabilities or limited English skills usually need more 
time to attain the same levels of competencies as other adult students. 

The inherent limitation of 1-year grant project periods can be construed as a disincentive to 
enroll Perkins participants in a career-ladder approach to education and training. Once again, this 
indicates that the Perkins Act's orientation is basically toward the secondary schools serving the 
traditional youth populations. This bias often prevents postsecondary institutions from promoting 
longer-term career training to the part-time adult learners and the at-risk populations that need it the 
most. 

As stated previously, obtaining an associate degree is not the only goal of community college 
students and should not be the only measurement used to evaluate success of Perkins programs at th 3 
postsecondary credit level. However, it should be an option available to all Perkins participants. 

Keeping the Dollars 

Accountability, both fiscal and programmatic, becomes a primary issue for monitoring and 
renewing grants, and for diversifying funding. 

The National Assessment could identify the working and n fiind-flow M relationships between an 
institution and its various funding sources to attain a true measure of the institution's efforts to 
promote integrated academic and vocational programing, especially for special populations. This data 
could be collected from institutional annual reports and audits of grant funding. 

ASSESSMENT DATA 

An effective National Assessment should address the progress of the new Act from an 
organizational development perspective. Change is difficult in the best of times as the process 
involves unfreezing the existing status quo, moving to a new condition, and establishing a new status 
quo. In order to accomplish this, the National Assessment needs to identify the sources of resistance 
to change, approaches used to reduce resistance, and the change agents. By using an organizational 
development model, the National Assessment will not only help determine how the funds are being 
spent, but why they are being spent in that way. Such a model will also identify impediments to 
progress. This information could be obtained by a self-report survey of the Perkins programs' project 
directors and institutional and state administrators. 

Obtaining relevant evaluation data at the end of a program is obviously directly related to the 
type of data collected from participants at the beginning of the program. Once the National 
Assessment is designed, institutions and regions will be able to modify their data collection systems 
and procedures to produce the types of data that are considered to be important in assessing the 
effectiveness of the new Act. It would be helpful to develop one common intake form that would 
meet the reporting requirements of the various funding sources committed to serving special 
populations and providing vocational training. 

Initially, an assessment design will necessarily need to center around the types of available 
dar At the postsecondary level, demographic profiles of the student body, enrollment data, retention 
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data, and completion data are readily available; however, to produce a valid analysis, they need to be 
coupled with the students' self-report surveys identifying reasons for enrollment. 

Followup job placement and wage data can probably be obtained from most postsecondary 
career placement offices. Future assessments may want to focus on employer satisfaction levels or 
reasons why employers hire applicants with specific types of training or educational backgrounds. 

Assessment documents that could be used at the postsecondary level include the following: 

• Surveys on demographic and labor market trends and demands of the specific 
geographic regions being served. 

• Demographic composition of the student body, including both credit and noncredit 
enrollments and completers. 

• Intake surveys for program participants which not only require necessary 
demographic data, but also ascertain the reasons for enrollment, in a forced-choice 
design format to determine baseline data for a value-added assessment. 

• Results of mail and telephone followup exit surveys to determine status of Perkins 
program completers. The best case scenario would be to conduct such surveys 6 
months after completion of training programs to address retention issues and to be 
consistent with existing JTPA, JOBS, and TRIO evaluation guidelines. 

• Recruitment, enrollment, and retention data comparisons of Perkins enrollments vs. 
overall postsecondary student body enrollments. 

• Case-management techniques and case studies. 

• Descriptions of institutional long-range planning processes and/or documents. 

• Fiscal analyses, including cost-per-student data. 

• Fiscal, programmatic, and data collection coordination with JTPA, JOBS, TRIO 
Programs, and other related funding sources. 

• Final narrative summative reports on Perkins programs from project directors, in £ 
standardized quantitative and qualitative format that allows for national comparisons. 

PROBLEMS TO AVOID 

Major problems in conducting the National Assessment can be avoided by involving local 
practitioners in the efforts from the beginning, by working closely with 2+2 tech-prep associate 
degree efforts being fostered by the American Association of Community and Junior Colleges, by 
accessing JTPA, JOBS, and TRIO Programs* assessment data and sources, and by accessing 
demographic and economic development data. Specific steps that can be taken include the following: 

• Involving postsecondary representation from the community college sector. 
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Involving representatives of the JTPA, JOBS, and TRIO programs. 



• Involving representatives of private industry. 

• Designing a standardized quantitative and qualitative final summative report form that 
can be locally distributed at the beginning of the grant period, rather than at the end 
of the project. 

• Redoing the 1988 NAVE Survey to identify the secondary-postsecondary distribution 
of Perkins funds and amending the survey to obtain the demographic and labor market 
trends in the geographic areas. 

• Encouraging the use of allocated monies for staffing to assist in evaluation/assessment 
efforts on all Perkins-fiinded programs. 

• Developing a "suggested" intake survey form compatible with the JTPA, JOBS, and 
TRIO programs' intake surveys for all students wishing to enroll in Perkins programs. 

In practice, when an institution receives funds to operate a grant project, it generally hires 
grant-funded staff to implement that project. Those staff will not necessarily still be with the 
institution when it comes time to conduct followup that extends beyond the grant project period. 
Such practical issues must be considered as the National Assessment is being designed. 

CONCLUSION 

In summary, some pitfalls should be noted regarding the National Assessment. The pr! r.vy 
danger is developing an assessment that is not practical to implement or is irrelevant to practice 
The second danger is developing an assessment that focuses exclusively on secondary education v&i 
disregards the role postsecondary education (i.e., community colleges) can and do play in 
implementing the Perkins Act. It is crucial to remember that the population is aging and we are no 
longer focusing only on the traditional youth population. Adult workers in need of retraining md 
upgrading should be considered as the emerging majority constituency to be served. 

The third danger is separating vocational education program planning from economic 
development, labor market issues, and demographic trends. 

A successful National Assessment will consider the basic constraints connected with the 
operation and administration of grant-funded programs and will allow for flexibility at the local level, 
emphasize the importance of direct service delivery to the students being served, promote interagency 
coordination, and measure success by value-added evaluations, using performance-based outcomes. 
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Assessing Academic Outcomes In Vocational Education < 

Thomas G. Sticht* 



INTRODUCTION 

Although wide variations have always existed in the thousands of schools that teach vocational 
education, a stereotypical account of such schools is that there are three tracks: one for the college- 
bound student, one for the general education student who may or may not want to go on to college, 
and one for the vocationally oriented student who, as typically characterized, prefers "hands-on" 
learning over "book learning," and is aiming to go to work, not college, after high school. 

This stereotypical, three-track system of education is still widely used. However, the results 
of the National Assessment of Vocational Education (NAVE) reported by John Wirt (1991) contradict 
the idea that there is strict separation between college-bound and work-bound (vocational education) 
students. The NAVE found that college-bound sophomores took an average of 3.2 Carnegie Units 
(over six semesters of work) in vocational education. This was over half the 6.0 number of units 
taken by vocational education students (students planning not to seek further education after high 
school but rather to find full-time work) (Wirt, 1991, p. 427). On average, about 20 percent of 
coursework by all students (not just those planning to go to work after high school) is vocational 
education. 

THE NEW SCENE: 

THE TRANSFORMATION OF VOCATIONAL WORK INTO ACADEMIC WORK 

Typically, vocational education students tend to perform lower than other students on National 
Assessment of Educational Progress (NAEP) and other academic achievement tests of reading, 
writing, and arithmetic skills and of academic knowledge (science, history, etc.). Until recently, this 
was acceptable given these students* interests in work, not school. Today, however, the picture is 
changing. 

Students who complete vocational education programs are still expected to acquire vocational 
knowledge and skill in using that knowledge in vocational activities. However, they now are 
expected also to acquire the knowledge and skill developed in "academic" programs (i.e., the 
programs usually taken by college-bound students). This is necessary, it has been argued, because the 
world of work, including blue-collar jobs, has become much more demanding, requiring higher levels 
of knowledge, thinking ability, and social skills. The idea that a child might drop out of school and 
get a pretty good paying job as a "shade tree" mechanic or a "seat of the pants" welder is inoperative. 

In today's shops, the mechanic's and the welder's tool kits come with extensive documents, 
technical manuals, and schematics that demand the ability to think abstractly and to imagine the 
outcomes for actions before they are carried out. Good eye-hand coordination is still required on the 
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tool and die makers' shop floors, but they are heavily aided by computer-generated designs, complex^ 
specification sheets, and programed operation sequences. The old motion of a blue-collar, "hands-on- 
job is rapidly being replaced by the notion of a "heads-on" job. 

Another factor pushing vocational education in the direction of providing both vocational and 
academic outcomes is the change of the economy from primarily blue-collar manufacturing jobs to 
white-collar services. In white-collar jobs, the ability to communicate well, in both oral and written 
language, is a key to success. Additionally, these jobs typically involve processing information, such 
as completing order forms, reading computer print-outs, preparing financial papers, writing reports of 
meetings and agreements, and a myriad of tasks involving manipulation of information in symbolic 
form. 

To a large extent, the idea that work requires "hands-on" skills and postsecondary study 
requires "heads-on" skills is no longer valid. Today, as suggested above, both students who are 
expecting to continue their education after high school and students who expect to enter the world of 
work after high school must be facile with the manipulation of symbolic information. This means that 
the traditional way of thinking abut college-bound and vocational education students needs to change. 
More and more, the world of work and the world of education are merging in their skill demands. 
The world of work is being transformed to look more like the world of education. 

ASSESSING ACADEMIC SKILLS IN NATIONAL ASSESSMENTS 

Hie merger of the aims and skills development needs of academic and vocational students has 
not made its way into our methods of assessing the skills development of secondary students. The 
major assessment of educational progress, the NAEP, does not assess secondary students* knowledge 
of vocational domains of work, even though about a fifth of the courses taken by college-bound 
students are vocational courses. Nor do these assessments measure students' academic abilities within 
vocational domains. 

For the most part, national educational assessments focus on assessing students' abilities to 
read, using textbook materials, news stories, poetry, or similar types of materials aimed at revealing 
how well students have developed large general bodies of knowledge and can use this knowledge to 
read and comprehend a compendium of materials. Similarly, assessments of writing are oriented 
towards writing narratives or expositions, featuring richness of description and expression of emotion. 
Sometimes a mundane "life-skill" such as writing a business letter is assessed, but this is not 
emphasized because it is not relevant to preparation of the college-bound student who will study 
classical literature and composition, and perhaps major or have a career in creative writing. 

In addition to assessing reading and writing of a general and academic nature, national 
assessments typically include other examinations that require students to use their reading and writing 
skills in content domains such as science, mathematics, history, geography, social science, and 
English. These assessments usually require that the students possess a fairly large body of facts, 
concepts, principles, and rules in each of these domains of knowledge if they are to perform the 
various tasks called for in a superior manner. 

Of these various content domains, mathematics is usually chosen as especially important 
domain to assess, with several important subdomains, including basic computations, pre-algebra, 
algebra, geometry, trigonometry, and calculus. This sequence of knowledge of mathematics follows a 
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typical academic progression of topics, and does not focus on the full use of mathematics in 
vocational and other aspects of life outside the school. It is the sequence that prepares students to 
reason abstractly in mathematics in a manner that will prepare them for further, college-level study of 
the subject. 

Limitations to Academic Assessments 

General versus job-related reading 

There is evidence to suggest that assessments that are confined to academic subjects may not 
provide an accurate indicator of how well students can perform vocational tasks. In research 
conducted for the U.S. Army, new recruits were administered a standardized reading test that 
provided an estimate of their reading skills in "reading grade levels'* (Sticht, 1975). For example, a 
person reading at the 5.5 grade level on the standardized test would score at the level of a student in 
the fifth month of the fifth grade of elementary school. The recruits were also given job-related 
reading task tests (JRTT). These were construct ed-response tests which included samples of materials 
taken from particular Army career fields, such as supply clerk and automotive repairman. The tests 
were based on materials and questions obtained by interviews with workers at their job sites. 

The general and job-related reading tasks were administered to two groups in addition to the 
unselected sample of recruits. One group was a sample of recruits who possessed special aptitude for 
the job field for which a particular JRTT was developed. For example, a sample of recruits with 
special aptitude in automotive and shop information were administered the general reading test and the 
automotive repairman's JRTT. Recruits with special aptitude for clerical work were administered the 
supply clerk's JRTT and the general reading tests. The other group who took the job-related reading 
and general reading tests comprised recruits who had special aptitude for the given career field and 
who had completed a technical training course in the job area. 

As indicated in Figure 1, for recruits given the supply clerk's JRTT and the automotive 
repairman's JRTT, the higher the general reading level, the better the performance on the JRTT. 
This confirms the value of high levels of reading skills for preparing students for the world of work. 
Of special importance, however, is the fact that, for the recruits with lower levels of general reading 
skills, those with special aptitude, or with special aptitude plus job training, performed better on the 
JRTT than they did on the general literacy test. 

Looking at the data for Supply Clerks (figure 1 A), we see that those reading at the 5.0 grade 
level who had special aptitude for clerical work scored about 40 percent correct. This is equivalent to 
the score of the general recruit population (those without special aptitude) with general reading skills 
at the 6.5 grade level. Similarly, the recruits with both special aptitude and training scored around 47 
percent correct, equivalent to the score of the general population with a general reading level at about 
the 7.5 grade level. Similar trends are found for the automotive repairman's data. 



117 

114 

ERLC 



Figure 1. Scores on job-raading task tasts by Army parsonnal with 
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The data in figure 1 show that even though recruits with special aptitude and training read at a 
general 5.0 grade level on the general reading test, they did considerably better on the vocationally 
related reading tests. If these recruits were assessed only with regard to their general reading level, 
using academic-oriented tests, their work-related skills would be underestimated by as much as one or 
two grade levels. They would do much better in their job-related reading than their general reading 
scores would suggest. 

The Armed Services Vocational Aptitude Battery. That assessments focusing on general 
academic skills may not accurately reflect people's vocationally related academic skills is also indicted 
by data from the Armed Services Vocational Aptitude Battery (ASVAB). 

The ASVAB contains 10 subtests (table 1) that range in generality from broad word 
knowledge (WK) (vocabulary) and paragraph comprehension (PC) reading tests, through mathematical 
knowledge tests of numerical operations (NO), arithmetic reasoning (AR), and mathematics 
knowledge (MK), to vocationally oriented tests of electronics information (EI), mechanical 
comprehension (MC), and automotive and shop (AS) information. 

As indicated in table 1, the " academic" word knowledge test is more highly correlated with 
other "academic" tests (paragraph comprehension, general science, arithmetic reasoning) than with the 
vocational tests of electronics information, mechanical comprehension, and automotive and shop 
information. Paragraph comprehension, a general reading test, correlates poorly with automotive and 
shop (and with other vocational tests). 

Table 1 

Intercorrelations Between ASVAB Subtests for 
Profile Study Staple 
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AR * Arithmetic Reasoning CS = Coding Speed 

WK * Word Knowledge AS * Auto and Shop Information 

PC * Paragraph Comprehension MK = Mathematics Knowledge 

NO * Numerical Operations MC * Mechanical Comprehension 

OS * General Science EI « Electronics Information 



The ASVAB data indicate that although there are positive correlations among academic and 
vocational knowledge, indicating that highly knowledgeable persons tend to possess both academic 
and vocational knowledge, these relationships are not strong. In particular, the less knowledgeable 
(as indicated by the data above) may possess more ability in a vocational domain than their general 
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academic skills would suggest. The armed services use these differences in academic and vocational 
knowledge to adjust their entry requirements for military service. At times, if a person's skills in 
general, academic areas (reading, mathematics) are too low for admittance into service, he or she may 
be admitted anyway if he or die has fairly high scores in one or more vocational areas (Eitelberg, 
Laurence, Waters, and Perelman, 1984). 

Vocational versus academic mathematics assessment 

In the most recent reporting on the mathematics achievement of U.S. school students, results 
are presented in the traditional academic manner, giving students' scores for topic areas including 
"numbers and operations," "estimation," "measurement," "geometry," "data analysis, statistics, and 
probability," and "algebra and functions" (Mullis, Dossey, Owen and Phillips, 1991). From such 
academic-oriented assessments, we are supposed to be able to make inferences about how well our 
students are prepared not only for college work but also for vocational employment. However, aside 
from a few "real-life" tasks, such as totaling the cost of a lunch, most items are brief and require 
tasks that few would perform in fields of work, even quite technical fields. For instance, one item 
requires students to draw a line parallel to another line that has a equation y =2x-5. The new line is 
to go through the origin 0. Then, the students are to write an equation for the new line. Most high 
school seniors could not do either of these tasks. 

While the assessment focused on mathematics as it is generally taught— as abstract subject 
matter— mathematics used in the world of work is sometimes much different. For instance, in a 
project for the U.S. Navy, a course was developed that called for the use of essentially 9th-grade 
mathematics in performing complex tasks. The tasks required that students read and comprehend 
detailed task instructions, determine the types of computations and systems of measure to use, and 
communicate their results using a variety of tabular and graphic formats. 

To accomplish these tasks, students were required to locate data tables in various chapters of 
a Navy textbook that included three sections: mathematics for managing financial resources, 
mathematics for managing human resources, and mathematics for managing material resources. 

Figure 2 illustrates the first task in the mathematics course. It shows that even though the 
mathematics required may be simple addition and subtraction, when embedded in complex tasks using 
multiple displays distributed at different locations in a text resource book, the amount of information 
processing is considerable. 
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Figurf 2. tuplt of tasks fro» Mavy-related lithtattici course. 



Chapter 2 
MATHXMATI C8 AND FINANCIAL RESOURCES 
Section 1 
Military Fay 

The following tasks tr§ related to the content of Chapter 1 "Pay" in the text: 
Mathematics for Navy Care er Progression . 

I. MILITARY FAY (A) 

1) POl Roberts was assigned to attend the tax advising class in January. 
He was supposed to answer questions members of the crew had about filing their 
income tax returns. After a week of classes to learn the current tax rules, 
he set up an afternoon training class to tell service members what he had 
learned. 



While getting ready for the class he found he needed several visual 
aids. Using the most recent Basic Pay chart he had, he drew up charts and 
overhead transparencies to show several things. 

Among the calculations he put on his displays were: 

a. The difference in monthly basic pay for an E-3 and E-4 under 2 
years service. 

b. The difference in monthly basic pay for an E-5 with under 2 and an 
E~5 with 4 years of service. 



c. The annual basic pay for an E-6 with 8 years service. 

d. Monthly basic pay plus BAQ for a married with 4 years of 
service* 



e. Annual Basic Pay and BAQ combined for a married E-5 with 5 years of 
service* 



f. The range of pay for E-5's with under 2 to 10 years of service. 

g. Monthly pay for an E-4 with 4 years (living in quarters off-base) 
without dependents, plus BAQ, minus 10% of his base pay. 

h* The percentage of increase in basic pay for an E-5 with 5 1/2 years 
of service who is advanced to E-6 seven months later. 



Make the calculations Roberts made for his displays. 



Specially developed tests were used to assess mathematics skills using decontextualized, 
academic formats such as columns of numbers to be added, subtracted, etc. In a separate section, 
vocationally oriented tasks were assigned which involved the same types of mathematics skills, 
contextualized within Navy tasks of the type illustrated in figure 2. Results indicated that even though 
students might make few mistakes using the decontexualized, academic assessments, when they were 
asked to perform more complex, contextualized information-processing tasks in which the same 
mathematics skills were embedded, they made more computational errors. 
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A COGNITIVE SCIENCE MODEL FOR ASSESSMENT 



The Army work on job-related reading and the ASVAB data illustrate the interrelations among 
bodies of knowledge and the information-processing skills used to learn and work with those bodies of 
knowledge. In the Army work, the fact that recruits with low levels of general literacy could perform 
better than expected on reading tests related to bodies of vocational knowledge they possessed 
illustrates that reading comprehension is not a "unitary" skill equally assessable by any test. Rather, 
the body of knowledge the reader possesses greatly influences the degree to which the reader can 
comprehend written tests. 

Similarly, the ASVAB test data indicate that even though all the tests require use of reading 
skills, individual's scores on the "academic" and "vocational" subtests will vary depending upon their 
background knowledge relative to the particular test. 

Both of these examples illustrate a model of human cognition in which bodies of knowledge 
are developed to which skill in using oral and written language are applied. Even mathematics tests 
involve reading the instructions, stems of questions, and completing the computational part of the 
task. Problems in performing on the assessments may reflect either lack of skill in the oral or written 
language or lack of knowledge in the domain(s) of knowledge being assessed. General education 
develops large bodies of knowledge in the traditional academic fields that are "addressable" using oral 
and written language. Thus, the NAEP science tests assess knowledge about scientific domains by 
administering a written language test. Therefore, the test requires both science knowledge and 
reading skill. 

In general, the finding that both knowledge and reading skill are involved in the NAEP 
content area tests suggests that such tests are simultaneously assessing reading and content knowledge, 
though they have not been optimally designed to reveal separate information about reading and 
content knowledge. Interestingly, even though reading and content knowledge are intimately related, 
there are extensive (and expensive) NAEP assessments of reading and writing in which the knowledge 
domains being read or written about are not identified. The assumption appears to be that if a student 
can read or write about one thing, she or he can read or write about anything. 

The foregoing suggests that reading can also be used to assess vocational domains of 
nowledge. As mentioned, the ASVAB does just this. It requires applicants for service to read about 
electronics, automotive and shop tools and procedures, etc. Further, just as vocational knowledge can 
be assessed using reading skills, reading skills can be assessed within vocational domains of 
knowledge. This is particularly important given the current interest in integrating vocational and 
academic education (discussed below). 

The Human Cognitive System 

In general, the points raised above suggest a model of the human cognitive system. This 
simple model includes long-term memory, which contains the knowledge base (along with language 
and various information-processing knowledge). When thinking is taking place, information 
processing occurs in short-term or working memory. The information-processing skills operate on 
information in the knowledge base and information acquired from knowledge bases "outside the 
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head/ such as books, speech, and nonlanguage sources. Information-processing skills may include 
various thinking skills as well as the communication skills using oral and written language. 

This simple model clarifies the relationships among content knowledge and information- 
processing skills such as reading, writing, reasoning, etc. For example, it indicates that reading 
comprehension can be improved by improving reading skill or by increasing the knowledge base 
about the subject matter— or both. 

The Navy mathematics project also illustrates that the load on working memory due to 
increased information-processing demands may disrupt other information processing, such as applying 
procedural knowledge about computation contained in the long-term memory. When task complexity 
is increased, as in performing the Navy-related mathematics tasks, working memory may be 
overloaded and errors may occur. 

Integrating Vocational and Academic Skills Training 

The current vocational education legislation calls for integrating vocational and academic 
education to enhance academic skills. For some, this has meant ensuring that vocational students take 
more academic courses, but it can also mean that vocational courses should emphasize development of 
academic skills (reading, writing, mathematics) while teaching vocational domains of knowledge. 

The Navy's mathematics course discussed above presents a case in which reading skills 
(comprehension strategies), computational and mathematical communications skills, and Navy 
vocational content (managing financial, human, and material resources) were taught in an integrated 
fashion. Assessment data from general reading tests, Navy-related reading tests, general mathematics 
tests, and Navy-related mathematics tests indicated improvements in all the test results, with the 
greatest improvement occurring in the Navy-related tests of reading and mathematics. 

This case study is significant because it illustrates that the academic skills of reading and 
mathematics can be developed within the context of a given educational domain with vocational 
content. And it is possible to assess separately the development of the body of vocational content 
knowledge and mathematics and reading skills, using vocationally related assessment tools. Also, by 
using "general 1 * test instruments, it is possible to obtain an indication of the extent to which use of 
mathematics knowledge and reading skills has been generalized. 

CURRENTLY AVAILABLE DATA FOR ASSESSING ACADEMIC SKILLS IN 

VOCATIONAL EDUCATION 

At the present time, there are no assessments designed to reveal differences between content 
knowledge (vocational or academic) and information-processing skills. Indeed, the NAEP tests have 
been built on a model of the school curriculum rather than on a coherent model of an educated human 
mind— the knowledge it should possess, the processes that should operate on the various bodies of 
knowledge, and how the mind should operate in various settings (home, school, and work). 
Furthermore, it appears that the model adopted for improving the academic skills of work-oriented 
students is the curriculum model for college-bound students. That is, academic skills improvement 
means getting vocational students to look like college-bound students on tests built for the latter. 
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Data Sources 

Data files with information on the academic outcomes of vocational students ate available in 
the various NAEP assessments of school children. In addition, the 1986 report of the NAEP adult 
literacy survey is significant because it assesses young adults* skills in performing N world-of-work N 
tasks with embedded reading, mathematics, and more complex information-processing skills (Kirsch 
and Jungeblut, 1986). Although this literacy profile does not assess specific content knowledge about 
vocational fields, it does permit comparison of the skills of college-bound and vocationally oriented 
students with skills of people in the work force in performing such tasks as using a bus schedule, 
reading a financial page, computing a checkbook balance, and other "real-life" tasks. 

The Defense Manpower Data Center (DMDC) contains millions of files on the ASVAB scores 
of applicants for military service and high school students who take the ASVAB for vocational 
counseling purposes. The ASVAB data are important because they include both academic outcomes 
assessment (reading, math, and general science) and technological literacy outcomes: electronics 
information, auto and shop information, mechanical comprehension, etc. This permits comparison of 
academic and technological literacy outcomes for both vocational and academic students. National 
norming of the ASVAB in 1980 utilized a subset of the National Longitudinal Study of 1976 
population, and information from that file could be useful. 

The New York State External Degree Program data files may contain information regarding 
the performance of vocational and academic students on competency-based assessments that utilize 
performance rating, portfolios, and exams. The American Council on Education (ACE) is currently 
developing a national version of the External Degree Program and may obtain information to compare 
vocational and academic students. The ACE also collects data on GED examinations; these files may 
contain information useful for assessing the academic outcomes of vocational students. 

WHAT ARE SOME PROBLEMS TO CONSIDER? 

While the data bases discussed above are available now (and new data will be added to them 
in the next few years), they do not address the problems discussed earlier related to assessment of the 
academic skills of reading, writing, oral communication, and mathematics within vocational domains. 
While they indicate, imperfectly, the degree to which vocational students resemble college-bound 
students on the various tests, they do not indicate how well college-bound and vocational students can 
perform on assessments of both vocational and academic knowledge and academic skills. 
Consequently, they may provide an unfair indication of the ability of lower-scoring vocationally 
oriented students to use academic skills in work settings. 

For the most part, the current assessments focus on the extent to which the world of work has 
come to resemble the world of school, without considering the opposite perspective. That is, they do 
not consider how the world of school could more closely resemble the world of work. 

The SCANS Work 

The perspective that considers how the world of school can resemble the world of work has 
been taken by the U.S. Department of Labor, Secretary's Commission on Achieving Necessary Skills 
(SCANS). In this commission's work, an attempt has been made to study the skill demands of 
occupational fields such as manufacturing, health services, retail trade, accommodations and food 



124 



/ 



service, and office services. In particular, attention has centered on identifying skills of high- 
performing businesses and industries in these occupational fields. The aim has been to identify for 
the schools the kinds of knowledge and skills graduates of the schools should have mastered to 
succeed in high-productivity businesses. 

In its work, the SCANS consulted cognitive scientists who, for the most part, draw (implicidy 
or explicitly) upon the human cognitive system model discussed above in planning, conducting, and 
interpreting their work. These scientists convinced the SCANS that academic skills always operate in 
the context of some domain of knowledge. Further, the application of knowledge and skill in a task 
context is a unified, inseparable activity. That is, while one can conceptually distinguish between 
reading and preparing a customer service report, functionally, reading is an integral part of the task 
and is influenced by other demands of the task, as in the Navy work on mathematics discussed above. 

For this reason, the SCANS attempted to produce scenarios in which the various tasks 
workers perform in their jobs are described as part of an overall work experience. Embedded in the 
scenarios are various bodies of knowledge and skills, called "know-how," that are common across 
various jobs and occupational fields. Figure 3 summarizes the "workplace know-how" determined to 
be needed by all school graduates if they are to participate productively in high-performance work 
environments. (Secretary's Commission on Achieving Necessary Skills, 1991, p. vii). 

In this scheme, the five competency areas are domains of knowledge used in all fields of 
work. For instance, all workers must use (manage) various resources such as time, money, materials, 
people, etc. They do this using the foundation skills: reading, writing, thinking creatively, 
reasoning, etc. Thus, though competencies and foundation skills are discussed separately, in use they 
are inseparable. 

One goal of the SCANS is to encourage schools to teach foundation and competency "know- 
how" to all students, both college-bound and vocationally oriented. The SCANS recommends that 
this know-how be developed by teaching students to perform tasks within the functional contexts of 
simulated or genuine work environments. The premise is that skills are best learned within a context 
that is meaningful to students, instead of in the decontextualized manner characteristic of academic 
education. 
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Figure 3* The Secretary of Labor's Commission on Achieving 
Necessary Skills (SCANS) lists of workplace "know-how" schools 
need to teach students to prepare than for work in high- 
performance businass and industrial. 

Workplace Know-How 

The know-how identified by SCANS is made up of five competencies and a three- 
part foundation of skills and personal qualities that are needed for solid job 
performance* These include; 

Competencies - effective workers can productively use: 

• Resources - allocating time, money, materials, space, and 
staff; 

• Interpersonal Skills - working on teams, teaching others, 
serving customers, leading, negotiating, and working wall with 
people from culturally diverse backgrounds; 

• Information - acquiring and evaluating data, organising and 
maintaining files, interpreting and communicating, and using 
computers to process information; 

• Systems - understanding social, organizational, and 
technological systems, monitoring and correcting performance, 
and designing or improving systems; 

• Technology - selecting equipment and tools, applying technology 
to specific tasks, and maintaining and troubleshooting 
technologies* 

The Foundation - competence requires: 

• Basic Skills - reading, writing, arithmetic and mathematics, 
speaking, and listening; 

• Thinking Skills - thinking creatively,, making decisions, 
solving problems, seeing things in the mind's eye, knowing how 
to learn, and reasoning; 

• Personal Qualities - individual responsibility, self-esteem, 

sociability, self -management, and integrity* 

Given the new focus on making the world of school more closely resemble the world of 
work, attention should be given to how new assessments can be developed to indicate how well 
secondary school students* foundation skills are being developed within the domains of both the 
traditional academic fields and the new workplace competencies. 
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The Success of Schooi-to-Work Transition 
James R. Stone III* 



"The high school's largest service is the best possible training for economic 
efficiency, good citizenship, and full and complete living for all its pupils." 

These thoughts were expressed by William Lewis around the turn of the century (cited in 
Spring, 1990). They highlight a recurrent theme: public education should contribute to students' 
future economic success. One manifestation of this concern is the increasing calls for developing a 
European-style apprenticeship system (Hamilton, 1989; Hoer, 1990; Lerman and Pouncy, 1990). 
This is not a new idea. 

"The German technical and trade schools are at once the admiration and fear of all countries. 
In the world's race for commercial supremacy we must copy and improve upon the German 
method of education. ..." (The Report of the Committee on Industrial Education of the 
National Association of Manufacturers (1905) cited in Spring, 1990.) 

More recently, recommendations have come from a variety of sources. The U.S. Department 
of Labor (1989), the Educational Testing Service (1990), and the National Center on Education and 
the Economy (1990) are among others arguing for American adaptations of the German model of 
apprenticeship. 

"At the age of 16, the average German— or Austrian or Swiss, or Danish— young person 
begins the transition to adulthood... At the age of 16, the average Arkansas— or New York or 
California— young person faces two more years of compulsory education that too often seems, 
and generally is, irrelevant to the world of work (Northdruft, 1990)." 

A second manifestation of the concern is expressed in language found in the most recent 
version of the Perkins Act, which calls for an evaluation of the academic and employment outcomes 
of vocational education, one of which is successful school-to-work transition. 

DEFINING SCHOOL-TO-WORK TRANSITION 

Traditionally, successful school-to-work transition has been conceptualized as the movement 
between the end of forma) secondary or postsecondary education and the acquisition of an entry-level 
job related to the vocational program— or the continuation of job-related training and education. 
While this is a reasonable expectation, this definition is labor market or demand-side dependent, and 
ignores the unpredictability of local labor markets, the increasing dynamism of the national labor 
market, and what Hamilton (1990) refers to as the "youth labor market." 



♦The author is Associate Professor, Department of Vocational and Technical Education, University 
of Minnesota and Project Director, National Center for Research in Vocational Education. 
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A modification of this definition that recognizes the increasing instability of labor markets 
would define successful school-to-work transition as a condition where the student can demonstrate 
the skills necessary for entry into a primary labor market occupation or career path consistent with his 
or her aspirations. These skills would include generic occupational skills, job-specific skills, and 
those skills required for successful adaptation to a changing labor market. This definition is supply- 
side oriented and focuses on the creation of employable graduates with a broad base of skills that will 
allow movement into a variety of occupations. Silberman (1982) argued: 

"...it is better for teachers to focus on attainable goals that are intrinsic to the educational 
process itself than on extrinsic goals such as employment and earnings over which they have 
little control. In this viewpoint, the primary purpose of vocational education is to promote 
full human development through exposure of the learner to activities that are intrinsically 
meaningful and absorbing. These activities may utilize various forms of work experience as a 
part of the educational process, but the purpose of the work is to further the education of the 
student; the work is subordinated to the educational process/ 

This definition differs from the traditional one on two key points. First, the focus is on 
preparing individuals for the primary labor market (see Hamilton, 1990) by focusing on the skills 
associated with such work. The second distinction is de-emphasis on initial labor market entry. This 
permits the emphasis of vocation education to be on that over which it has some control. 

This paper has four objectives. The first is to place the problem of school-to-work transition 
in its proper context by examining the sources of the problem. The second objective is to examine 
present and possible perspectives on what school-to-work transition means. The third objective is to 
discuss the appropriate foci for the National Assessment's investigation. The fourth objective is to 
examine related methodological issues and existing and possible sources of data. The paper concludes 
with an examination of existing pedagogy that enhances the school -to- work transition. 

THE CONTEXT OF THE PROBLEM 

A convincing argument can be made that we do a very poor job of preparing our youth to 
make the transition from the adolescent role of student to the adult role of productive worker. At 
best, we prepare some to continue their education, and we hope for the best for the rest— a hope that 
is increasingly unfulfilled. 

In comparison with our European competitors, we do the worst job of facilitating the 
transition from school to work (Tucker, 1990). As noted, German, Swiss, and Norwegian young 
people begin the transition from school to adulthood at age 16, and work through what has been 
described as a "gentle process** by which workplace learning gradually supplants classroom learning 
(Youth Apprenticeship, American Style, 1990). 

By contrast, 16-year-olds in the United States face another 2 years of public school that for 
many are boring and irrelevant to their world. The vast majority, between 66 percent and 89 percent, 
will begin working part-time (and some nearly full-time) to pay for their short-term aspirations: a car, 
$100 Reeboks, and their social life. And they work at jobs that have little or no connection to their 
schoolwork. 
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Two important consequences result from this behavior. The first is that adolescents may 
develop negative work attitudes as they continue milling around in secondary labor market jobs (see 
Greenberger and Steinberg, 1986). Further, because the students see no relationship between their 
out-of-school work and their in-school work, there is little possibility that these experiences will 
expand their vision of future career possibilities. 

The second consequence of this behavior is that young adults will remain in the status of 
"adolescent worker - until they are in their mid-twenties, when they might "settle down" and are 
presumed to be mature enough for primary labor market jobs (Shanker, 1990). The effect is that we 
prolong adolescence for a decade longer than our European counterparts. This is a serious waste of 
scarce natural resources. 

"...our economy is being damaged and more importantly, young lives are being damaged, by 
our collective failure to help young people make a smoother transition from school to work 
(William T. Grant Foundation Commission on Work, Family, and Citizenship, 1988). 

The problems created by an inadequate school -to-work transition result from at least three 
additional contextual factors: before-school factors, in-school factors, and after-school factors. 

Before-School Factors 

Youth entering high school today bring with them the effects of a variety of social conditions 
that influence their performance in school. An increasing number of children, currently more than 
one in four, live in poverty. Fourteen out of 100 students are children of teen-aged mothers. One in 
four students suffers from drug-related problems (Minnesota Federation of Teachers, 1989). At least 
2 million school-aged children have no adult supervision after school. In 1988, 40 percent of users of 
shelters for the homeless were children (Hodgkinson, 1991). 

In short, our classrooms are increasingly being filled with the kind of children whose 
problems we usually ignored in the past or whom we were content to "throw away." We called them 
"dropouts," but in fact many were pushed out of school because of our unwillingness to provide the 
help they needed. This has led to what some have called "social concentration camps without walls." 
As a society, we can no longer afford to be this short-sighted. 

In-School Factors 

The system of public education has created a discontinuity between school and work. This is 
a result of its singular emphasis on preparing students for college. As Parnell (1990) observed, "Our 
education system has allowed only one standard of achievement— the college-prep baccalaureate 
degree program— to predominate." Education is disconnected from the world of work, he argued. 
He also declared that if this condition is allowed to continue, the United States will find itself in 
serious trouble. 

In addition to a myopic focus on preparing individuals for traditional college tracks, schools 
have two related shortcomings: high school pedagogy and the focus of high school counseling. 
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What we teach 



Part of the transition problem area arises from what schools do and do not teach. Many 
argue that schools are not teaching the "right stuff," and as a consequence, students are not motivated 
to work hard. This leads to a situation where students emerge from school unskilled and 
unmotivated. An unfortunate conundrum is highlighted by Gunderson (1990) when he argues that, 
"students complain about schools; schools complain that business is not supporting them; businesses 
complain because they do not feel that the educational system is doing what is necessary and they see 
themselves doing more and more to qualify people for work" (p. 4). We are left with a situation 
where students, perhaps a majority, do not believe that what they do in school makes any difference 
and where business does not value what the high schools provide. 

Career counseling 

While nearly 40 percent of American high school graduates enroll in some form of college 
after high school, a sizable percentage do not complete college (Hoyt, 1990). Charner (1990) 
estimated that only 25 percent of these students will complete a 4-year college degree. Thus, 
counseling for non-college careers is important. However, most students lack adequate career 
counseling. As Johnson (1990) notes, educators are career illiterate. Most school guidance 
counselors have advisee loads of 300 or more students and do little more than take care of problems 
and find college application forms for requesting students. Barton (1991) observed that almost half of 
all high school students said they had never talked to a counselor about occupations. 

After-School Factors 

The third problem area is what happens after school. Adolescents in our society spend 
several years moving around in the workplace (or sometimes at school) until they are perceived as 
mature enough to hold down adult jobs. Age becomes a proxy for reliability. In the process, we 
have created a youth labor market (Hamilton, 1990). These jobs, according to Hamilton, are most 
likely to be in retail, food service, clerical, and unskilled manual work. They rarely require a high 
school diploma and typically do not require any training beyond a short demonstration of the job 
tasks. 

Instead of moving into real, meaningful work as most European youth do by age 18, 
American youth are condemned to years of pursuing work in a severely constrained labor market 
where "advancement" is obtained by changing jobs. As Shanker (1990) noted, firms wait to hire 
applicants until they are 24- to 25-years old to see how they do elsewhere. For many other 
employers, a college degree become a proxy for maturity. 

We are also faced with the problem of those students who do not survive the process of public 
education. We do little for the out-of-school youth in our country. It has been estimated that we 
spend $5,000-$6,000 on the post-high-school education of a youth who continues formal schooling 
and less than $500 on one who does not continue. 
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PRESENT AND POSSIBLE PERSPECTIVES 
ON SCHOOL-TO-WORK TRANSITION 



The current perception of school-to-work transition is nested in a paradigm of vocational 
education that has its origins in the politics and social structures of the 19th century. The paradigm 
was most commonly known as "the doctrine of social efficiency/ This led to a definition of the 
successful vocational student as one who can perform specific tasks identified with an industry or 
occupation. The effect of this was to look at successful school-to-work transition in terms of a count 
of the number of students employed in the "field for which trained." 

The problem with this approach is that it ignores the reality of today's workplace. Unlike any 
other time in our history, workers must be able to adapt to increasingly rapid changes in their jobs. 
At the same time, more workers will be subject to voluntary and involuntary job changes during their 
working lifetimes. Some estimates place the number of major career changes at seven or eight during 
an individual's working life (Bernard, 1991). 

As a result, it makes little sense to assume that vocational education has been successful if a 
graduate can find an entry-le H job related to a narrow set of skills identified with a particular 
program. This does not mean that these skills are unimportant. However, it makes more sense to 
assume that, in addition to learning a set of specified skills useful in the present job, a successful 
graduate can demonstrate the kinds of skills which will ensure that he or she will be able to adapt to 
the constant change the future seems certain to offer. 

Successful school-to-work transition is more than placement statistics or wage data. Knowing 
how to weld using current technology is a necessary but not sufficient condition of future 
employment. Knowing how to weld and knowing the principles of welding and being able to 
comprehend and adapt to new technologies yet to be discovered are the necessary and sufficient 
conditions for successful transition from school to work and for future transitions in the labor market. 

THE FOCUS OF ASSESSING SCHOOL-TO-WORK TRANSITION: 

WHAT TO MEASURE 

The current Perkins Act requires that states adopt at least one measure of learning and 
competency gains in academic skills and at least one measure of competency gains in behaviors 
related to labor market success (i.e., job- or work-skill attainment, competency attainment, retention 
in school, or job placement) (Hoachlander, 1991). These requirements suggest both short-term and 
long-term assessment strategies. 

Short-Term Assessment * 

Short-term assessment of school-to-work transition should focus on the preparation for 
transition. Assessing the structural aspects of the program and student abilities are the twin 
dimensions of this focus. 

Vocational programs should be assessed on the presence and quality of specific structures 
linked in the literature to successful school-to-work transition. A list would include the following (see 
Bishop, 1989; McKinney and Halasz, 1981): 
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• supervised work experience component 

• placement services upon program completion 

• linkage between academic and vocational studies 

• early and continuing career counseling 

• connecting adolescent workers with adult mentors 

• concentration of coursework around a single vocational area 

• quality of related jobs or occupational experiences 

• characteristics of cooperating firms providing training 

Generally, assessment strategies focused on vocational participants should identify the 
potential for transferring school-based learning to work settings* The specific performance measures 
should focus on the extent to which vocational education develops both knowledge of technology and 
the process skills of work-based learning that will be the key to individual and corporate success. 

The knowledge of current technology and the ability to learn and adapt to new technology will 
be a key dement iri successful school-to-work transition. In addition, Carnevale (1990) has identified 
a list of process skills, or "upskilled worker traits/ found to be necessary for the emerging labor 
market. The study should include quantitative and qualitative assessments of vocational program 
participants* 

• Ability to transfer school-based learning to work settings 

• Capacity to learn how to continue learning in a work setting 

• Reading levels 

• Teamwork skills 

• Writing and computation levels 

• Interpersonal communication skills 

• Problem-solving abilities 

• Personal management skills 

• Self-esteem 

• Leadership abilities 

• Negotiation skills 
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Some of these upskilled worker traits can be measured by behaviors available through an 
analysis of school records. Such behaviors include attendance and tardiness patterns, participation in 
vocational organizations and leadership within those organizations. Other behaviors and attitudes 
related to upskilled worker traits include dimensions that might be measured through employer 
assessments (where appropriate) and through self-assessments by students. These quantitative 
assessments would include (Barton, 1991) 

• Ability to use basic skills in a work environment 

• Students' success in integrating school and work learning 

• Subsequent occupational socialization 

A second, short-term dimension of school-to-work transition has been discussed by Dole 
(1990) and Hoachlander (1991). They identified the sort of school-related outcomes one should 
associate with participation in vocational education. These included 

• Gains in academic achievement 

• Program completion rates 

• School completion rates 

• Successful completion of higher level vocational courses 

• Completion of higher level academic courses 

• Subsequent pursuit of related education or training 

• Acquisition of the specific skills necessary to obtain employment in the students* area 
of occupational interest 

Long-Term Assessment 

The long-term assessment should focus on patterns of transition. This would include the 
following: 

• Patterns of entry into and progress through primary labor market jobs, including 

- self-employment 

- patterns of earnings 

- duration of employment 

- job satisfaction and job satisfactoriness 

- sustained employment in a primary labor market occupation 
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* job quality of post-school employment 



• Participation in subsequent education and training 

• The extent to which employers recognized the vocational program completion as a 
creditable credential 

• Subsequent use of school-developed, work-related skills 

METHODOLOGICAL ISSUES 

Defining the Vocational Student 

A continual problem in assessing the effects of vocational education participation is defining a 
vocational participant. It has been known for some time that most high school students take some 
vocational education— perhaps as many as 79 percent (Campbell, Gardner, and Seitz, 1982). Meyer 
(1981) found considerable disagreement between individual and school classifications of vocational 
student status. Stone (1985) speculated that the 30 percent disagreement shown in the literature 
probably results in the exclusion of "academic" students from vocational assessments that rely on 
school classification of status. 

The best strategy to account for this would be to follow Meyer's (1981) advice and use 
continuous measures of vocational participation available through school transcripts. It would also be 
useful to control for other causes of post-high-school attainment, such as the extent to which students 
concentrated their coursework in a given vocational area, other kinds of vocational participation, and 
(he amount of advanced coursework included in their school experience (see Stone, 1988). 

There are a number of research issues surrounding assessment of school-to-work transition. 
The following is a brief list of issues related to the use of existing data bases for post-hoc analyses 
and to the generation of new data through qualitative processes. 

Existing Data Sources 

The use of national data bases may obscure the quality factors within programs that lead to 
successful school-to-work transition. It is not valid to assume that all business education programs or 
all electronics programs are equally well structured or equally well taught. This problem might be 
overcome if existing (particularly the National Education Longitudinal Survey, 1988) and future 
educational surveys can incorporate questions that would assess the quality of the various vocational 
programs and the results of participation. 

There are a number of data sources that could provide information for assessing the short- 
term and long-term impact of vocational education participation on school-to-work transition. Two 
sources of data would be useful for these purposes: national longitudinal studies and national 
recordkeeping systems. 
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National longitudinal studies 



Although there are scores of national longitudinal studies available, four offer a set of 
variables that will address some of the issues related to school-to-work transition. 

Learning Through School-Based Work-Experience Programs (LSBWE). This 5-year 
study was initiated in 1988 and was funded through the National Center for Research in Vocational 
Education. The focus of this study was to answer a series of questions about work-experience 
programs regarding the effect of working while in school. Specifically, the study was designed to 
examine the effect of the quantity and quality of work experience on educational, economic, social, 
and psychological student outcomes and to determine how the school might arrange an environment to 
amplify the desirable effects of work experience. 

The sample includes approximately 3,000 high school and community college students located 
in six purposively selected sites. Data were collected from students, their employers, school records, 
and instructors/coordinators of school-supervised work experience programs. Students completed an 
annual fbllowup. Categories of variables include 

1. Past and present labor market experience 

2. Educational experience and aspirations 

3. Work attitudes and behaviors 

4. Job quality characteristics 

5. Socioeconomic indicators 

6. Social/psychological indicators 

The National Longitudinal Study of Youth (NLSY). This is one of a series of five national 
longitudinal surveys of labor market experience sponsored by the U.S. Department of Labor. These 
studies were designed to monitor the labor market experiences of groups which were of special 
concern to policymakers. 

The original national probability sample consisted of 5,578 females and 5,828 males who 
were between the ages of 14 and 21 in 1978. The sampling procedure included an oversampling of 
minorities. 

Data were collected directly from students, through interviews with family units, and from 
school records (transcripts). Students completed an annual fbllowup. Categories of variables 
included 

1. Past and present labor market experience 

2. Job characteristics 

3. Educational experiences and aspirations 

137 



ERLC 



136 



4. Social/psychological indicators 

5. Socioeconomic indicators 

6. Local labor market conditions 

7. Deviant school and social behaviors 

High School and Beyond: A National Longitudinal Study for the 1980s. This longitudinal 
study was designed to examine the impact of in-school activities on postsecondary educational 
attainment. A secondary purpose was to capture changes that had occurred in education-related and 
more general social conditions since the 1970s. 

The study design included a highly stratified national probability sample of over 1,100 high 
schools. This yielded a sample of 30,000 students of the class of 1982. In addition to the student 
surveys, transcripts were obtained for approximately half of the students. Included in the data are 
variables that measured 

• Vocabulary, math, and reading ability 

• Educational experiences and aspirations 

• Social/psychological indicators 

• Social status indicators 

• Work experience in high school 

• Post-high-school labor market experience 

National Educational Longitudinal Study of 1988 (NELS 88). The NELS 88 study is part 
of a program of longitudinal studies sponsored by the National Center for Education Statistics. Other 
studies in this program include the National Longitudinal Study of the High School Class of 1972 and 
the High School and Beyond longitudinal study. 

The purpose of NELS 88 was to provide a data base that would identify school attributes 
associated with achievement, the transition of different groups from eighth grade to secondary school, 
the influence of ability grouping on future educational achievement, and changes in educational 
practice over time. Data were collected from students, school records, parents, and teachers. In 
addition, the design calls for area vocational school data augmentation as well as vocational-technical 
institute augmentation. 

National recordkeeping systems 

WORKLINK is a joint initiative by the National Alliance of Business, the National Urban 
League, and the Educational Testing Service. It is a recordkeeping system linking education and 
training with business and industry. Initially (1990), WORKLINK programs are being field-tested in 
two sites (Indianapolis and Tampa). 
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New Data Needed for Assessment 

While it is likely that existing sources of data will provide much useful information for the 
assessment of the success of vocational education in facilitating school-to-work transition, there are 
questions of concern that are not addressed. At least two strategies exist to overcome this problem. 

As noted earlier, questions of interest can be added to current national longitudinal studies 
such as the NELS 88. While the initial focus of NELS 88 is on within-school transition, it offers the 
best potential for including additional questions of interest to the current National Assessment since 
the students in this cohort will not be expected to graduate until 1992. A bank of questions focused 
on the school-to-work transition discussed in this paper could be added while the students are still in 
school or on the first followup survey. 

However, not all of the questions or issues posed in this paper lend themselves to quantitative 
assessment. One strategy that holds promise for helping to understand how participation in a 
vocational program facilitates school-to-work transition is the case study method. 

The case study method envisioned for the National Assessment would require initial validation 
of the criteria for assessing school-to-work transition. Using these criteria, nominations of exemplary 
schools or programs could be sought through state school officers. A number of these "outliers" 
could be selected to reflect different communities or types of students served. The purpose of these 
case studies would be to identify the conditions or circumstances that lead to exemplary performance. 
A secondary purpose would be to identify what is replicable in other settings. 

ENHANCING SCHOOL-TO-WORK TRANSITION 

The ability to learn in the workplace is a function of vocational program quality, the quality 
of the related work or simulation experience, and the extent to which these two are related. On the 
job, quality can be assessed by examining the nature of the training, the characteristics of the work 
environment (e.g., autonomy, problem-solving opportunities, etc.), and the philosophy of the training 
sponsor. 

In the classroom, vocational program quality can be assessed by examining the pedagogy to 
determine the extent to which use is made of the real experience of work to develop occupation- 
related skills and behaviors and advance the use of basic skills. Historically, four forms of pedagogy 
have been used for this purpose: apprenticeships; simulations, shops, or labs; cooperative education; 
and school-based enterprises. 

Apprenticeships match individuals with "masters' 1 of a trade and combine school-based 
training with on-the-job training. Usually, formal agreements are signed by all parties. However, the 
average apprentice in the United States is a person in his or her late 20*s. In their present structure 
and practice, apprenticeships do not offer much in the way of school-to-work transition for 
adolescents or young adults. 

Simulations, shops, or labs use simulated work in simulated settings to achieve the goals of 
the curriculum. Although they are commonly used in vocational education, they are arguably the 
least effective form of pedagogy for facilitating school-to-work transition. By contrast, existing 



139 



135 



evidence on cooperative education (and similar evidence evolving for school-based enterprises) 
suggests the usefulness of these forms of pedagogy in facilitating school-to-work transition. 

Cooperative vocational education uses real, productive work in an actual work setting to 
advance the goals of the curriculum. School -based enterprises use real, productive work in controlled 
settings for the same purpose. These forms of pedagogy are discussed below. 

Cooperative Vocational Education 

Cooperative vocational education (co-op) has a long history in the United States as a method 
for school-to-work transition. Programs that use the co-op method place students in jobs related to 
classes they are taking, and develop agreements and plans for linking classroom instruction with on- 
the-job training. 

Even though programs that use this method provide a direct school-to-work link and have 
demonstrated value in facilitating school-to-work transition, few students participate in these 
programs. Recent figures from the High School and Beyond Longitudinal Study of the High School 
Class of 1982 indicate that only 7.8 percent of students participate in cooperative education programs. 
More recent evidence would suggest that participation is decreasing at a time when more students are 
working during high school. Thus, fewer students are able to formally connect their work with their 
schooling. 

School-Based Enterprises 

In school-based enterprises, students work in, and often run, enterprises that create real 
products and services that are consumed by other students or the general public. There are many 
examples of successful enterprises in the literature (see Stone, 1990). They typically include daycare 
centers, retail stores, florticulture shops, automobile repair services, and credit unions. In some 
instances, schools take over defunct businesses in their community. Some enterprises are the result of 
partnerships with local businesses, others result from the ingenuity of the vocational instructor or 
other educational leader. 

The full extent to which school-based enterprises are used is not well known. At present 
there have been no formal surveys of practice that might shed light on this useful form of learning. 

CONCLUSION 

The National Assessment's investigation of school-to-work transition should consider both 
short-term strategies and long-term strategies. The short-term assessment should focus on vocational 
education participants 9 potential for moving successfully from the school setting to the workplace and 
the programs 9 success in preparing them for that move. Long-term assessment should look at 
patterns of entry and progress in primary labor market jobs. An assessment with these two 
dimensions should yield useful information for both practitioners and policymakers. 
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The Relevance of Vocational Education 
for Subsequent Employment 



Thomas Bailey* 



INTRODUCTION 

In drafting the 1990 Perkins Act, Congress was concerned that the education students receive 
in vocational courses be "relevant" to their subsequent employment. But what does relevance mean 
and how would we measure it? Does it refer to the direct transfer of specific techniques or pieces of 
information, to more general abilities or know-how, or to broad attitudes and behaviors? What if 
vocational graduates get good jobs outside of the specific areas for which they are trained? Does that 
necessarily mean that their education was irrelevant to subsequent employment? 

Like so many seemingly simple questions, the one regarding vocational courses' relevance to 
employment remains simple only if we fail to think about it seriously. The question is difficult to 
answer because the answer depends on the ultimate purpose of vocational education. This purpose 
not only is difficult to define but also is changing. In fact, we can view the 1990 Perkins Act as an 
attempt to change the purpose of vocational education. As the model changes, the way we think 
about relevance— particularly the way we measure and assess it— must also change. 

In this paper, three alternative models of vocational education are identified. Within the 
context of each model, the relevance of vocational training to subsequent employment is discussed. 
Based on the discussion, suggestions are offered for the design of the National Assessment of 
Vocational Education called for by the 1990 Perkins Act. 

THREE MODELS OF VOCATIONAL EDUCATION 

1. Preparation for a specific occupation: In the first model, vocational education is 
preparation for specific occupations. This type of education is aimed at students who plan to 
enter the labor market immediately after high school or community college and the programs 
are specifically designed to prepare them for a particular occupation. 

2. Broad occupational preparation for nonbaccalaureate students: In the second model, 
vocational education is designed to provide broad occupational preparation and a basic 
educational foundation for nonbaccalaureate students. This model emphasizes maturation of 
nonacademic students, career exploration, building some type of connection to the labor 
market or to post-school activities enjoyed by college or college-bound students, and training 
in broad occupational areas. 



*The author teaches at Columbia University where he is Associate Professor of Economics, Teachers 
College and Associate Research Scholar, Conservation of Human Resources. 

145 



ERLC 



140 



3. A foundation for broad educational reform: In the third model, vocational education is a 
crucial building block for fundamental restructuring of education for all students. This 
approach is based on an educational strategy that rejects the distinction between vocational and 
academic education and advocates applied education that integrates vocational and academic 
instruction for all students, including those bound for and enrolled in 4-year baccalaureate 
education. 

The 1990 Perkins Act and the associated reform movement can be viewed as an attempt to 
move from the first model (preparation for a specific occupation) to the second model (broader 
occupational preparation for nonbaccalaureate students). Much of the discussion and research related 
to broader educational reform supports the third model, but it is unlikely that the Perkins Act will be 
the primary vehicle through which such a reform will come about. The amendments do emphasize 
integration of academic and vocational education, but early experience suggests that this will be seen 
as a reform of vocational education rather than of education in general. Academic material probably 
will be added to traditional vocational programs, but it is unlikely that much effort will be made to 
incorporate vocational instruction into traditional academic programs. 

Model 1: Specific Occupational Preparation 

If the purpose of vocational education is to prepare students for particular occupations, it is 
simple enough to ask whether students end up in those occupations. This concept has resulted in a 
measure referred to as the "training-related placement rate" (TRP). The last National Assessment of 
Vocational Education (NAVE) did some very interesting work in this area, making important 
improvements in the concept. Previous work had been based either on school- or self-reported 
student categorizations as vocational, academic, or general students and, among vocational students,, 
as participating in programs for particular occupations. However, analysis of student transcripts 
indicated that self-reported categorizations often did not match school-reported categorizations and that 
these often did not bear obvious relationships to courses and course-sequences taken by the students. 
Indeed, vocational coursetaking in both high schools and community colleges was distributed more or 
less evenly, and almost all secondary students took at least one vocational course. Therefore, it made 
more sense to talk about students taking more or less vocational instruction than it did to place them 
in vocational, academic, or general student categories. 

By using student transcripts, the NAVE staff developed a "course utilization rate" (CUR). 1 
This measure was based on determining the relevance of individual courses to particular occupations. 
The work by NAVE used a course/occupation crosswalk developed by the National Occupational 
Information Coordinating Committee (NOICC). 2 When a student's post-school occupation was 
categorized, a count was made of the courses the student took which served as preparation for the 
occupation. The ratio of this number to total vocational courses was the CUR. The NAVE staff 
further refined this measure by excluding all occupations for which no preparation was necessary, 



l The following discussion of the NAVE findings is based on U.S. Department of Education Fina* 
Report (1989). 

2 For a description of the NOICC crosswalk and changes made by the staff of the previous National 
Assessment of Vocational Education see Horn and Meyer (1990). 
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arguing that it made little sense to measure the relevance of a course to an occupation that needed no 
formal preparation. (The following discussion refers only to the "skilled jobs CUR.") 

What did the NAVE research find? Using the sophomore cohort of the High School and 
Beyond (HS&B) data set (most of these students graduated from high school in the spring of 1982), 
the research found that by 1985, 44 percent of the vocational courses taken by students in the sample 
were relevant to their subsequent employment, up from 38 percent in 1983. Thus, 3 years after 
graduation, the students* education appeared to be more relevant to their employment than it had 1 
year after graduation. 

There was a sharp contrast between men and women. The CUR for women was 53 percent 
in 1985 and 46 percent in 1983, while the rate for men had risen from 32 to 38 percent over the same 
period. Women had rates of 65 percent for business (more than one-half of all courses taken were in 
the business field) and 79 percent in health-related occupations (only 3 percent of the courses taken 
were in this field). The CURs for both business and health rose between 1983 and 1985. For 
women, the only other large area was occupational home economics, accounting for 18 percent of the 
courses. However, the CUR for this occupational sector actually fell from 51 to 32 percent between 
1983 and 1985. For men, the highest CURs occurred in three traditional male occupational 
areas— agriculture (43 percent), construction (42 percent), and mechanics and repairers (45.5 percent). 

In summary, findings showed that rates were higher for women, both men and women had 
their highest CURs in occupations traditionally dominated by workers of their own sex, and CURs 
rose as graduation receded. 

CURs were higher for postsecondary students. About 58 percent of vocational courses were 
used in subsequent employment. Only about 53 percent of courses taken in proprietary schools were 
used, while 60 percent of courses taken in community colleges and public technical institutes were 
used. The authors concluded that "students of different races and socioeconomic status who take 
similar amounts of training, choose similar fields, and enroll in similar types of institutions, are likely 
to experience similar economic outcomes [the use of vocational training in subsequent employment]" 

(p.no). 

The NAVE report also analyzed the reasons for low course utilization. The report concluded 
that the problem was not that in the aggregate too many students were trained for given occupations. 
Rather, students voluntarily chose to work in other occupations or perhaps were unable to find 
training-related jobs in their geographic areas. Thus, there did not seem to be a large overall 
mismatch between training and employment areas. However, there may have been some local 
mismatches (e.g., students were not able to find the openings that existed or changed their minds, 
perhaps having been unsure when they originally chose their fields). One policy implication is the 
need to improve counseling, placement, and local planning. 

This NAVE research lays the foundation for some interesting subsequent work. One area 
involves further examination of the causes of variation in the CURs and of reasons why students do 
not end up in occupations for which they trained. The methodology used by the earlier NAVE study 
to examine the causes of low utilization is highly inferential and can be considered as only a first step. 
We will not have a good answer without more direct information, probably from interviews or 
appropriate surveys of completers. 
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The crosswalk itself is another area that could use additional work. Clearly, the results can 
change dramatically, depending on how broadly we interpret the skills learned in a particular course. 
CURs can be raised simply by using a broader definition of skills learned in particular courses. For 
similar reasons, it is difficult to determine a normative standard against which to judge the level of a 
CUR. Is 50 percent high or low? This normative judgement depends on the nature of the crosswalk, 
the breadth of the skill definitions, and the underlying model of vocational education being used. 
Indeed, these conceptual problems have been made more difficult by changes in vocational education 
called for by the 1990 Perkins Act. 

Model 2: Broad Occupational Preparation 

During recent decades, traditional vocational education has frequently been criticized for 
providing training appropriate only for narrowly defined occupations. As the skill needs of the 
economy were perceived to change more rapidly, vocational education, especially at the secondary 
level, appeared to provide training for jobs that would soon be obsolete. Low training-related 
placements or course utilization rates also suggested that narrowly defined vocational education was 
misguided. Training that was useful only in one occupation was wasted if the student did not work in 
that occupation, which was the case for many students who took vocational courses or even fall 
sequences of vocational education. Moreover, the central educational reform thrust of the 1980s 
called for increasing academic course requirements for high school graduation. In many states, this 
restricted the time available for vocational courses. While reformers may not have set out to shrink 
vocational education, it was an outcome many did not oppose. The belief that vocational education 
was a thinly disguised tracking system or a "dumping ground" for students who could not make it in 
the more desirable academic tracks further weakened the support for vocational education. 

Whatever the ultimate merit of these ideas, the 1990 Perkins Act seeks to broaden the 
education received by students in vocational programs. Thus, the Act encourages both integration of 
academic and vocational instruction and teaching much broader information about the industries. For 
example, instruction in carpentry would include strengthened academic material as well as 
components or courses that teach construction design, project planning and management, legal issues, 
and other relevant topics. While students would be prepared to go to work as carpenters, they would 
also have both a solid grounding in the basic academic areas and some background in many related 
occupations. This improved vocationalism would serve to motivate many vocational education 
students and attract others who were not interested in straight academics. The work-related focus 
would also provide natural ties to the business world and would help connect students to post-school 
work. Furthermore, research (including some by the earlier NAVE) suggests that for some students, 
standard academic material is taught more effectively in applied courses (Meyer, 1989). 

In the second model, the focus on the particular occupation begins to blur. Nonbaccalaureate 
students will indeed be prepared for post-education work, but they will also have solid preparation for 
a wide variety of related work and will be better prepared to acquire new skills as technology and 
work processes change. 

What implications does this have for evaluating the relevance of vocational education for 
subsequent employment? One conclusion is that it confiises the interpretation of measures such as 
training-related placements or course utilization rates. If the purpose of the current reforms is to 
weaken the links between in-school instruction and particular occupations, measuring the extent to 
which students end up in particular occupations no longer measures the extent to which goals of 
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education are met. For example, if the goal of vocational courses is to help teach basic academic 
skills, those courses will presumably be useful in any occupation where basic academic skills are 
used. In addition, if students learn about aspects of an entire industry, placement in any job in that 
industry might be a related placement. The broader education presumably will teach students 
managerial, planning, organizational, problem-solving, and interpersonal skills that will be of use in 
many industries and occupations. For example, agricultural vocational education and related 
programs such as Future Farmers of America are often cited as exemplary vocational models because 
they prepare young people for a wide variety of leadership roles outside of farming. From this 
perspective, the more successful the program, the lower the apparent relevance of the courses if a 
narrow measure such as the TRP or CUR is used. 

At the very least, full implementation of the reforms called for in the 1990 Perkins Act would 
require a thorough revision of the crosswalk that links courses to particular occupations. If the old 
measures are used, then paradoxically, successful implementation of the spirit of the 1990 Perkins Act 
may lead to a decline in CURs as students take advantage of the wider range of opportunities opened 
to them by the greater breadth of their education. Alternatively, if the definitions of course relevance 
are widened (e.g., to include all jobs in an industry), the CURs will rise. However, that rise will not 
necessarily yield information about whether the new, broader education is more effective in opening 
opportunities than the traditional approach. 

In terms of assessing skill relevance, the use of broader measures not tied to the increasingly 
vague association between particular courses or sequences and occupations or jobs is indicated* Two 
strategies can be outlined. One defines a set of generic skills and measures the progress students 
make in those skills while they are still in school. The second relies on general labor market 
measures such as income, compensation, wages, or employment to evaluate the effectiveness of 
vocational education. 

The first approach has the advantage of not requiring data on post-school employment. 
Another advantage in the short run is that the U.S. Department of Labor, Secretary's Commission on 
Achieving Necessary Skills (SCANS), is already identifying a list of generic skills. These skills are 
to be defined in a way that will make assessment possible. However, this approach depends entirely 
on the validity of the skill definition and the assessment method, and it completely abandons any 
attempt to evaluate directly the relevance of particular programs or courses to particular jobs or 
occupations. Of course, SCANS-like lists of skills could be developed for different occupations, but 
this would be a very large task that would require constant updating. In any case, it would lead back 
to linkage between course content and a particular occupation. 

The use of more general labor market outcomes (e.g., wage, compensation, or employment) 
provides a measure of the effectiveness of a particular program or course without having to establish 
a priori a link between the education and the job. Indeed, there is already a considerable literature 
that uses wages to evaluate vocational education. Some of these studies suggest that even traditional 
vocational education measures up well when compared to the alternatives. For example, a 1987 study 
by Campbell and his colleagues found that after controlling for a variety of characteristics including 
past and present enrollment in higher education, vocational graduates were about 8 percent more 
likely to be in the labor force and 3.5 percentage points less likely to be unemployed than graduates 
from academic programs. Pay for vocational students was higher as well. In contrast, an earlier 
study by Rumberger and Daymont (1982) found slightly higher unemployment and lower pay for 
vocational students. 
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Research using high school transcripts has allowed a more subtle analysis of the effects of 
different mixes of courses. Thus, Kang and Bishop (1989) found that for those students who did not 
attend college full time after high school, and 

...who took four full-year vocational courses and eight full-year academic courses in their 
final 3 years earned substantially more, immediately after graduating, than students who took 
12 academic courses. Business programs, for example, raised the earnings of females by 
$1,940 or 40 percent and trade/technical programs raised the earnings of males by $1,536 or 
22 percent. Once a student has taken four vocational courses, further increases in vocational 
coursework at the expense of academic courses have little or no positive effect on earnings 
immediately after graduation (p. 113). 

It is possible to argue that educators should not be concerned with whether vocational students 
end up in particular occupations, but should be concerned that the education the students receive is 
better than (or at least as good as) the alternative types of education are in achieving students* broad 
life goals. 3 While income and earnings are approximate measures of the achievement of life goals, 
their importance cannot be denied. 

There are problems with the wage or compensation analysis, as well as serious 
methodological issues. (See the paper by David Stevens included in this publication.) One problem 
involves how to treat students who go to college directly after high school. Limiting samples to 
students who go to work after high school can be misleading. In addition, a general analysis of wage 
or compensation changes resulting from vocational education does not pinpoint any of the causes of 
those economic changes. Even if it were possible to show that some vocational programs had strong 
positive effects, unless the underlying causes are quantifiable, it is unlikely that standard earnings or 
wage analysis will identify those causes. Without that information, policy planning and replication 
are thwarted. (A similar problem is encountered in analysis of the effects of teachers on student test 
scores. A great deal of research suggests that teachers can have a strong influence, but to date, 
researchers have been unable to identify which teacher characteristics are relevant.) 

Finally, wage- or labor-market outcome analysis takes time. The effects of today's policy 
will emerge over several years; therefore, no analysis initiated by the current National Assessment 
could be used to evaluate the effects of the 1990 Perkins Act. 

Despite these problems, analysis of labor market outcomes should be an important part of any 
assessment. Over the last 20 years, social scientists, including those involved with the previous 
NAVE, have made progress on many of the methodological problems that plague this type of 
research. The current National Assessment could continue to contribute to that progress. While new 
data will be of only limited usefulness for a report written in 2 or 3 years, the current assessment 
could make an important contribution by helping strengthen the available data for future analysis. 



3 From the point of view of educational policy, it is important also to take account of the cost of 
vocational education. Since vocational education usually costs more than academic education, it may not 
be cost-effective even if its graduates do as well as they would have in other programs. 
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Model 3: A Foundation for Broad Educational Reform 

Although the Perkins Act calls for integration of academic and vocational education, it is 
almost inevitable that these changes will be seen as reforms of traditional vocational education. That 
is to say, more academic material will be introduced into many vocational courses, but the 
introduction of vocational-like material into standard academic courses will be rare. After all, 
vocational education has a particular institutional and political history which sets it apart from 
education in general. Vocational educators have their own professional associations and lobbyists; 
there is a separate national center for vocational education research; and vocational programs are 
usually administered by separate departments of local school bureaucracies which report to separate 
agencies at the state level. The Perkins Act itself is an expression of the uniqueness of vocational 
education. Education is a state and local function, but through the Perkins Act, the federal 
government articulates its particular interest in vocational education. 

Despite the Act's focus on vocational education, much of the pedagogic research and theory 
that supports proposals in the Act can be applied to all students rather than only to those who 
traditionally take full vocational sequences, see themselves as vocational students, or are classified by 
their schools as vocational students. For example, one influential position argues that successful 
performance in the classroom is only weakly related to successful performance at work or in society 
(Resnick, 1988; Scribner,1988). Others argue that even the most basic skills are learned more 
effectively when they are taught in the context in which they are used (Sticht, 1989). Some 
evaluation research indicates that there are benefits for many students in using applied material to 
teach abstract concepts (Meyer, 1989). 

This line of thinking has led to a variety of related reforms that, in general, attempt to 
connect schooling more closely to nonschool activities. The reforms include a renewed interest in 
models of apprenticeship, cooperative education, and school-based enterprises (Stern, 1990; Stern et 
al., 1990). Advocates of expanded apprenticeship programs in this country point out that about 70 
percent of all German youth (starting at age 16) pass through apprenticeship programs in which most 
learning takes place on the job and the content of the classroom sessions students attend one day a 
week is coordinated with activities on the job (Hamilton, 1990). 

Advocates of the models of reform have also called for a variety of alternatives to the 
traditional paper and pencil approach to assessment. They include proposals for portfolio and project- 
based assessment and attempts to devise tests that can assess a student's abilities to solve complex 
problems (Berryman, 1990). 

Vocational education, especially if it is broadened through some of the programs now being 
encouraged by the Perkins Act, lends itself to many of these reforms. But the crucial political issue 
concerns whether students, parents, educators, and policymakers believe that a more applied education 
is appropriate for students headed toward traditional baccalaureate degrees. In fact, it is unlikely that 
one would find much evidence of these kinds of reform in the nation's most prestigious public and 
private schools. 

There are two problems related to the bias against general educational reform. First, it is 
possible that baccalaureate-bound students would be better served by a more applied and less abstract 
education. After all, SAT scores have been falling, even among the best students, and the high 
correlation between social class background and the apparent quality of the schools suggests that the 

151 

14G 



9 

ERLC 



perceived success of these schools may have more to do with resources at home than with any special 
value added by the school. Second, as long as vocational education (even reformed and renamed) is 
seen as most appropriate for students who will not or cannot go on to 4-year colleges, it will be very 
difficult to avoid making it a track for "second class citizens." 

If we take the broader view in which vocational education serves as a cornerstone of general 
educational reform, the traditional approaches to assessing the relevance of vocational education to 
subsequent employment would yield more misleading results. This further increases the value of 
broader measures such as those based on well-designed skills assessments or eventual labor market 
outcomes as opposed to direct matching of courses or sequences to occupations. 

Since the Perkins Act does not call for a general education reform, the National Assessment is 
not in a position to assess the effectiveness of such a reform. Nevertheless, it would be worthwhile 
for the National Assessment to develop some pilot projects to study the effectiveness of more applied 
education for baccalaureate-bound students. There are a few occupationally oriented schools 
explicitly designed to prepare their students for 4-year colleges, including one of the high school 
academies in Philadelphia and Lane Tech in Chicago. Lane is considered the best school in Chicago; 
it continues to require shop courses for all students. Studying the experience of college-bound 
students in cooperative education programs would also be valuable. The political context that shapes 
vocational education would be transformed if the current National Assessment produced research 
which suggested that reformed vocational education provided a solid preparation for selective 4-year 
colleges. 

CONCLUSION 

Based on this discussion, the following six suggestions are offered for the design of the 
current National Assessment of Vocational Education as it seeks to understand the relevance of 
vocational education for subsequent employment. 

1. Whatever work the National Assessment does, it should explicitly identify the underlying 
model of vocational education on which the analysis is based. The author has argued that 
normative measures of "relevance" would change as the model changes. This is particularly 
important at this time since the 1990 Act in effect seeks to change the model on which 
vocational education is based. 

2. More work is needed on the methodology for matching courses or sequences to occupations 
and jobs. Measures of "relevance" can change significantly simply by narrowing or 
broadening the conception of skills taught within a given course. At the same time, additional 
work on the occupation-course crosswalk should only be undertaken if the assessment staff is 
convinced that the conceptual basis of the crosswalk is consistent with the model of vocational 
education they are using. 

3. Research on why so many vocational students do not end up in occupations for which they 
were presumably trained would yield a great deal of useful information. (Some work on this 
could be done with the longitudinal data sets, but some small-scale surveys of graduates of 
particular schools might be necessary.) This line of research could not only lead to some 
insights about how vocational education interacts with the labor market, it might also help any 
redesign of the crosswalk. 
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4. As the Perkins Act moves vocational education from a model that emphasizes specific 
occupational training (Model 1), to one that emphasizes broader occupational preparation 
(Model 2), broader measures of "relevance" need to be developed. Currently, there is a great 
deal of research and discussion on different means of measuring and assessing school 
outcomes. The National Assessment should plan its activities to take advantage of this work. 
Because of its explicit focus on the skill needs of the workplace, the work of the Secretary's 
Commission on Achieving Necessary Skills might be particularly relevant. 

5. Also as a result of increasing emphasis on Model 2, measures of labor market outcomes will 
be an increasingly important tool for assessing the relevance of vocational education. 
Although there are many thorny methodological problems with this line of research, progress 
can be made. The National Assessment can help confront those problems both 
methodologically and by working to improve the available data. 

6. Finally, as a way to begin considering the merits of the third model— vocational education as 
the basis for broad educ^ional reform— the National Assessment should develop some pilot 
projects that would exa. une the relevance of reformed vocational education as preparation for 
baccalaureate education in 4-year colleges. 
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Occupations and Earnings of 
Former Vocational Education Students: 
Research Design Issues 

David W, Stevens* 



The Carl D. Perkins Vocational and Applied Technology Act of 1990, Sec. 403. (b)(5) 
requires a description and evaluation of "...employment outcomes of vocational education,.../ This 
paper translates this phrase from the Act into a research question; describes the principal issues that 
must be resolved in the development of a research design to answer the question; explores the 
availability of data to implement the design; and lists the most important constraints that will limit the 
researchers* ultimate success in responding to Congress' mandate. 

THE RESEARCH QUESTION 

The principal research question is: Are former vocational education students working in 
jobs that require the competencies they acquired through vocational education's auspices? 

Decomposing this question into its parts reveals the researcher's challenge: 

• What definition of "former vocational education student" should be adopted? No one 
definition will meet all evaluation requirements. Multiple definitions must be adopted. 
These should be based on the following criteria: vocational course intensity (i.e., 
measures of exposure to vocational education); student achievement (i.e., measures of 
enrollee accomplishment from this exposure); student characteristics (e.g., member of 
a particular special population); institutional characteristics (e g., secondary 
comprehensive, secondary vocational, community college, or community-based 
organization); and local area characteristics (e.g., levels of and changes in industry- 
specific employment). 

• How can we determine whether a job requires certain competencies? It is not feasible 
to attempt a direct determination of required competencies. We know that skill 
competency requirements vary among firms that produce the same product or service. 
We also know that skill competency requirements are often flexible depending upon 
the presence of other positive or negative screening considerations. Inevitably, the 
evaluator will be forced to rely upon existing occupational classification schemes 
(e.g., the Dictionary of Occupational Titles and related crosswalks to other 
classifications). 

• How can vocational education's contribution to acquired competencies be isolated? 
Again, the evaluator will not have any direct measure of vocational education's 



*The author is Director of the Regicnal Employment Dynamics Center, Robert G. Merrick 
School of Business, University of Baltimore (Md.). 

157 



ERLC 



151 



contribution. Attempts should be made to introduce available comparison group 
information to permit indirect estimates to be derived. 

EVALUATION DESIGN ISSUES 

Judith Gueron, President of the Manpower Demonstration Research Corporation, recently 
described the evaluator's task in the following way (Evaluation Forum, 1991, 76): 

Much of what evaluators can do to improve the use of their 'product' in the 
policymaking process depends on how credible, clear, and useful evaluation 
information is. At bottom, we need to have a good product. Then we need to make 
our conclusions accessible and be able to say what they should (and should not) 
suggest to policymakers and practitioners.... At the early stages, we also need to be 
thinking about what policymakers and practitioners can use. 

There is abundant evidence that the answer to the question, "Are former vocational education 
students working in jobs that require the competencies they acquired through vocational education's 
auspices/ will be: Some former vocational education students are working in jobs that require 
competencies they acquired through vocational education auspices; some are not. The members of 
Congress surely seek to learn more than this by mandating a description and evaluation of 
employment outcomes of vocational education. 

In 1994, reauthorization deliberations will include an intense review of 

• the consequences of allocating federal ftinds based on concentrations of low-income 
households; 

• the success of tech-prep; 

• the effects of changes in state and local administrative responsibilities; and, 

• the extent to which academic and vocational offerings have been integrated. 

For information to be useful in 1994, the design of an employment outcomes evaluation 
should reflect these known needs. However, this part of the National Assessment's overall research 
strategy should not concentrate an the 1990 Act's net impact on employment outcomes. The final 
National Assessment report mus* be submitted in 1993. Only mid-1992 employment information can 
be included in this report. These data cannot reflect much impact attributable to the 1990 Act. 

The evaluation design should reflect well-known interest in 

• training-related placement of former vocational education students; 

• distinctions among type and levels of vocational education institutions; 

• the importance of different concentrations of vocational education exposure; 
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• particular subgroups of former vocational education students (e.g., members of special 
populations); 

• isolation of influences on employment outcomes that are protected by administrative 
control from the effect of external forces; and, 

• the stability of findings and conclusions when different time intervals are considered. 

The evaluation design should permit investigation of both unusually positive and negative 
outcome patterns. Advantage should be taken of any opportunity to document the consequences of 
change. The evaluation should not respond to the unasked question: Should there be vocational 
education? Instead, it should reveal under what circumstances vocational education makes a 
difference. Distinctions among federal, state, local and private sources of funds should be left to 
other parts of the overall evaluation design. Here, the primary goal should be to detect differences in 
the employment outcomes of former vocational education students. A secondary objective should be 
to trace these differences to their causal origins. 

There is no consensus about what we should expect an evaluation of employment outcomes to 
reveal (Gustman and Steinmeier, 1980; Meyer, 1991; Stevens, 1983). If occupational skills are the 
principal determinant of employment opportunity, and if these skills are provided by vocational 
education, then we might expect to find high rates of training-related placement. But, if personal 
attributes and site-specific circumstances combine with occupational skills as determinants of 
employment opportunity, then we would expect to observe different rates of training-related 
placement for those with comparable skills. The research design must either choose among 
competing theories about what matters, or be flexible enough to permit a test of these competing 
theories. 

This is not only an academic issue. Arbitrary training-related placement rate standards are 
used throughout the nation to justify life-or-death decisions about vocational education programs. The 
evaluation design should promise to advance our understanding of whether such standards should be 
applied, and, if so, how they should be managed. 

Six evaluation design questions are of particular importance: 

1. How should the training-related placement issue be handled, given the difficulties 
predecessors have had in devising a satisfactory approach? 

2. Does it matter whether we use wage-rate, earnings, or compensation information? 

3. Are previous and concurrent employment and earnings histories of vocational 
education students relevant, or will subsequent employment and earnings information 
suffice? 

4. Are differences among institutional providers of vocational education pertinent? 

5. How should unit-of-analysis choices be made? 
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6. What is the relative importance of knowing more group average performance versus 
discovery of exemplary and abysmal outcomes? 

The Training-Related Placement Issue 

Vocational educators have traditionally sought to determine whether a former student in a 
well-defined vocational specialty went to work in an equally well-defined job utilizing the specialty's 
skills. It is now well known that most enrollees in vocational courses do not complete a sequence of 
courses that constitutes a uniform specialized program. It is less well known that the pervasiveness of 
narrowly defined occupations is also eroding. 

The evaluation design issue here is how to match vocational education exposure with 
employment application. Meyer (1991) uses course utilization rate and skilled job course utilization 
rate concepts for this purpose. The course utilization rate "... measures the share of all vocational 
courses taken by students that are related to jobs that they eventually obtain"; and the skilled jobs 
course utilization rate "... measures the share of all vocational courses that are related to the jobs that 
the students obtain only when those jobs require more than minimal skills...." 

Meyer's approach, which is the current state-of-the-art, adopts vocational course as a practical 
unit-of analysis, compares each of these courses with subsequent jobs held on an all-or-nothing basis 
(i.e., either related or not related), and then combines these courses to derive a utilization rate. 

This procedure has certain limitations. It 

• masks within-course differences in the skill acquisition of individual students; 

• ignores complementarities and duplication in course content, and the effects of these 
on student achievement; 

• necessarily relies upon available vocational course and occupational classifications for 
matching purposes; and, 

• does not distinguish between skill relatedness, skill requirement and skill sufficiency. 
A skill can be related but optional; required but not sufficient; or sufficient, which 
implies requirement and relatedness. Many people speak in terms of requirement, but 
appeal to measures of relatedness alone for accountability purposes. 

Each of these features of Meyer's method limits the practical management usefulness of 
reported findings. For example, if a course offers required skills, but personal attributes or site- 
specific circumstances are overriding determinants of employment opportunity, then the expected 
employment outcome should be modified accordingly. This is easier to assert than to accomplish. 
The Job Training Partnership Act performance standards record offers a valuable lesson from which 
vocational educators can benefit in their own approach to an inevitably contentious topic (King, 1987; 
National Commission for Employment Policy, 1988; National Commission for Employment Policy, 
1991). Local Private Industry Councils are urged to first determine whom they want to serve and 
then see how the choices they make affect expected performance. In fact, because of 
misunderstanding and skepticism about the criteria-specific adjustments, many of the local 
administrative bodies attempt to "play" the adjustments to their advantage. Substantial attention must 
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be given to staff training and consensus building about the appropriateness of the adjustments before 
they are introduced. The legislative and administrative history of Florida's statewide placement-rate 
standard for vocational education programs offers valuable lessons for other jurisdictions (Florida 
Employment and Training Placement Information Program, 1991). 

Another example of the limits of this method is that if either vocational course or occupational 
classifications are inaccurate descriptors, then the management usefulness of a knife-edge 
determination of related/unrelated must be questioned. Also, if within-course differences in 
achievement and course complementarities matter, then admission criteria and retention patterns are 
important. 

We know that each of the hypothetical cases stated above occurs some of the time everywhere 
and all of the time in some places. Therefore, available vocational course and occupational 
classification systems must be understood before committing to a particular evaluation design. The 
National Occupational Information Coordinating Committee, its member Executive Departments, 
affiliated State Occupational Information Coordinating Committees, and their constituent agencies, 
have collaborated to make tremendous progress in the development of crosswalks that permit 
translation from one classification to another. However, it remains the evaluator's responsibility to 
understand how these translations among special-purpose classifications affect the interpretation of 
observed employment outcomes. This problem is particularly acute given the proliferation of unique 
vocational education program codes that followed die widespread introduction of new program 
approval practices that were intended to diminish duplication of offerings. 

The U.S. Employment Service in the U.S. Department of Labor has appointed an advisory 
panel on revision of the Dictionary of Occupational Titles , which is to offer its recommendations in 
mid-1992. The evaluation design should reflect what is known about anticipated future refinements of 
current classifications. These changes are unlikely to affect the National Assessment evaluation per 
se, but they can be useful in fine-tuning the evaluation with an eye to post-assessment monitoring of 
employment outcomes. 

Wage Rate, Earnings, And Compensation 

There are two schools of thought among evaluators about the link between the importance of 
an outcome and the measurement of that outcome. Some argue that we measure what is important, 
while others say that we attribute importance to what has been measured. Money is considered by 
many to be a uniform measuring rod of the reward for work. This widespread agreement does not 
preclude disagreement about the relative importance of this measure vis a vis other rewards (e.g., job 
satisfaction, opportunity to continue to learn on the job, and promotional outlook). Many of today's 
evaluations of vocational education continue the precedent of using wage rate as an outcome measure. 
The merits of this practice are questioned here. 

Wage-rate information alone is a poor indicator of vocational education's success. Once, in 
the distant past, an evaluator could multiply a self-reported hourly wage rate by an unobserved but 
uniform full-time work week to obtain a reasonable estimate of earnings, which in turn was a reliable 
proxy for the total payoff from work effort. However, because of the disintegration of the uniform 
full-time work week into multiple definitions of "full-time," the traditional practice of assuming a 
uniform work week is unacceptable. Also, as a result of collective bargaining negotiations, a growing 
wedge has been driven between money earnings and total compensation, which includes the cash 

161 

lb6 

O 

ERLC 



value of nonwage benefits. Since participation in these benefits is uneven, continued reliance on the 
wage portion of total compensation alone compromises the accuracy of the intended measure of 
reward to work effort. 

Wage rate is an imprecise measure. For example, a high entry-level wage might indicate 
many things other than a more "successful" job, including 

• a straightforward measure of currently realized productivity; 

• a measure of anticipated future productivity that will be realized after successful 
completion of on-the-job training; 

• a dangerous or otherwise unattractive job; 

• an opportunity that is insulated from the leveling influence of competitive pressures; 
or, 

• a substitute for nonwage benefits. 

Further deterioration of the earnings estimate emerges when increasingly common voluntary 
and involuntary part-time employment are factored in. Simultaneous employment in more than one 
job, the routine use of mandatory overtime, shift differentials, commissions, bonuses, and self* 
employment add to the destruction of what was once a nearly universal "transparent" outcome 
measure. The evaluator's task is much more complex today than it was in previous decades. 

Previous, Concurrent, and Subsequent Employment 

In post-World War II America, it made sense to focus on the transition of secondary 
vocational education students into entry-level jobs upon graduation. In 1991, many vocational 
education students are enrolled in postsecondary courses, which means they are likely to have 
previous and concurrent employment histories. This creates a number of evaluation challenges. 

Many enrollees in vocational courses do not complete an uninterrupted and uniform sequence 
of courses that constitutes a specialty culminating in a diploma, certificate, or postsecondary degree 
(Stevens, 1991). New entrants into the work force, re-entrants who have been absent for some time, 
and those already in the work force who are renewing or enhancing their skills, are quite different 
from a management or policy standpoint. It is credible to argue that new entrants acquired their 
occupational skills in a vocational education program. This credibility is weakened when students are 
employed while they go to school or if the students have previously worked. 

Many former vocational education students have prior and concurrent work histories. The 
observed employment success for former vocational education students may be an ambiguous guide 
for offering future admissions advice, even if employment opportunities remain unchanged. These 
former students may already have been employed, enrolled in vocational education courses, and then 
remained with or returned to the same employer or moved to a new employer. The enrollment may 
have been recreational or job related, and it may complement or be unrelated to previous vocational 
education enrollment. The evaluator must consider at the outset what uses might be made of reported 
information about the employment of former students. 
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In the face of this diversity, careful thought must be given to how the proposed evaluation 
design will open and close doors to collecting and interpreting data. This will require appropriate 
sharing of information about proposed evaluation designs among those who will be responsible for 
specific parts of the overall National Assessment. 

Multiple Institutional Sources of Vocational Education 

The growing diversity of vocational education providers parallels that of vocational education 
students. The nation has witnessed a progression of institutional types, beginning with worksite 
apprenticeships, through secondary and postsecondary public schools, to today's mix of exclusively 
vocational public schools, combined academic and vocational public schools, private for-profit career 
schools, nonprofit community-based organizations, and worksite training offered through an 
employer's auspices or in collaboration with an external public or private vendor. 

These differences should not be overlooked in the design of an evaluation. Employment 
outcomes are likely to vary depending upon the vocational education provider's institutional 
characteristics. Some exclusively vocational settings are highly selective and prestigious; others are 
"dumping grounds'* that are equally well known but stigmatized. Particular institutional settings 
reveal high concentrations of one sex or ethnic group. Some depend heavily on funding sources that 
impose income-eligibility criteria. These distinctions must be recognized in the evaluation design if 
many management and policy uses of the evaluation's findings are to be promoted. 

Unit-Of-Analysis Choices 

No one group, especially not the Congress, is responsible for vocational education. The 
evaluation product will lie untouched on the shelf if it describes vocational education in generic terms; 
there is no potential buyer for such a product. There are many potential buyers of an evaluation 
product that investigates the employment and earnings outcomes of former vocational education 
students in districts with concentrations of low-income households, or in communities that have 
experienced dramatic realignments of the local economy, or in schools with high concentrations of 
minority enrollees, or in vocational course sequences with concentrations of a single sex, or in 
vocational programs that practice competency-based certification, or in states that apply performance 
standards in a management oversight context, or in communities that have multiple deliverers of 
vocational education, etc. 

A decision rule for evaluation design should be: Is there at least one identifiable management 
agent whose actions might be affected by the expected findings if the evaluation is undertaken in the 
proposed way? If the answer to this questions is no, the evaluator should go back to the drawing 
board. If the answer is yes, and more than one such agent can be identified, the evaluator must 
decide whether choices should be made among these potential consumers of the evaluation 
findings—can the relevance of the evaluation be improved for one or more potential users, but at the 
expense of other applications? This question sets the stage for introducing die last of the six design 
issues. 

Average, Exemplary, and Abysmal Outcomes 

Precise estimates of average performance are likely to be accepted with limited enthusiasm by 
many members of Congress. For example, there are vigorous dissenters (Swanson, 1991) from 
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provisions of the 1990 Perkins Act that concentrate federal funds on special populations. This should 
alert the evaluator to the importance of being able to describe vocational education's role in preparing 
students in unexceptional circumstances for productive and rewarding employment. 

However, vocational education is already in place and future management decisions will occur 
at the margins. It would be impractical to design an evaluation that would only answer the question: 
Should vocational education courses be offered? It is quite practical to design an assessment that is 
intended to discover unusual patterns of employment success or failure by former vocational education 
students, and to investigate these aberrations to reveal consistent correlates of this abnormal 
performance. 

The National Assessment's principal evaluation responsibility is to the Congress. The 
assessment design should favor federal interests in employment outcomes. Can the evaluator detect 
employment patterns that offer the Congress opportunities to fine tune the federal role in vocational 
education? Do seemingly typical former students move on to atypical employment affiliations? If so, 
what can be said about the reasons for this? Among the obvious former student populations that will 
be of interest to the Congress in 1994 are special populations, local areas with high concentrations of 
low-income households, states that use explicit performance standards, and completers and early 
leavers in tech-prep sequences. 

Credible conclusions based on these units-of-analysis will be useful to the Congress in 1994 
reauthorization deliberations. Other evidence pertaining to employment outcomes in unexceptional 
settings will be useful in defining the tone of the debate. Credible evidence of widespread 
employment of former vocational education students in jobs that require the competencies they 
acquired through vocational education will support a positive tone. Contrary evidence, or evidence of 
questionable credibility, will establish a negative tone in 1994. The credibility of the evaluation's 
findings will depend heavily on the evaluation design's balance in permitting favorable and 
unfavorable evidence to emerge. The Congress will receive sufficient anecdotal evidence from 
vocational education's supporters and adversaries. The National Assessment evaluation design should 
be unassailable by either group. 

The evaluator's design task is similar to that of an experienced mountain climber who 
confronts a previously untested challenge. There may be many possible ways to traverse the terrain, 
but previous experience contributes to a quick reduction of the practical number of routes to be 
considered. Then, the cumulative consequences of each decision at a branching point must be thought 
through. This strategic thinking exercise is intended to produce a preferred approach (Dixit and 
Nalebuff, 1991). 

Finally, vocational education goals, strategies, and circumstances differ within and among the 
federal, state, and local management levels. The evaluation design should reflect these differences. 
This will help insulate the National Assessment's findings from criticism. 

AVAILABILITY OF USEFUL DATA FOR EVALUATION PURPOSES 

Desired data items can be listed based on the evaluation design issues that were examined in 
the previous section. They include: 1) individual attributes and transcript and employment history 
information for former vocational education students; and 2) information about local area economic 
conditions and changes in these conditions. 
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The availability of this information for the universe of enrollees in identifiable vocational 
education courses, concentrations of courses, and institutions will strengthen the evaluation. 
Appropriately chosen samples can suffice, but the adequacy of a sampling design is often difficult to 
convey to lay observers. In fact, the availability of information on an entire population of former 
students will permit the selection of different samples to determine what effect this has on reported 
employment outcomes. 

Included among the many forces that affect employment outcomes and should be measured are 
job availability, qualification for the available jobs, awareness of these jobs, and willingness to accept 
and work in these jobs. Each of these factors is a contentious issue between vocational educators and 
their critics. Whenever possible, the evaluation should account for differences in these forces. 

There is no nationally representative data source that contains all these data. The cost and 
length of time necessary to create such a data set rule this out as a viable option. Locally based 
followup reports are of such uneven quality that they cannot serve as a reliable basis for producing a 
credible report on employment outcomes for former vocational education students. 

On the other hand, some of the states have taken advantage of the plunge in cost and 
tremendous increase in capability of automated management information systems to develop more 
comprehensive and standardized employment data sets. These are the National Assessment's most 
promising source of useful information about employment outcomes for former vocational education 
students. 

State Employment Security Agency Data 

The Deficit Reduction Act of 1984 mandated that "...employers in [each] State are required, 
effective September 30, 1988, to make quarterly wage reports to a State agency (which may be the 
agency administering the State's unemployment compensation law)..." (Stevens, 1989). Three-fourths 
of the states were already collecting this information at that time through State Employment Security 
Agencies in support of their state-administered unemployment compensation programs. Today, only 
Massachusetts and New York collect the required information through an agency other than the State 
Employment Security Agency. 

Among the states that have used these State Employment Security Agency data for vocational 
education followup purposes are Arizona, Colorado, Florida, Kansas, Maryland, Missouri, Utah, 
Virginia, Washington, and Wisconsin. Two of these states, Washington and Wisconsin, have 
investigated former postsecondary student outcomes only (Strong and Jarosik, 1989; Washington State 
Board for Community College Education, 1990). To the best of the author's knowledge, Florida is 
the only state that maintains an automated statewide secondary and postsecondary placement 
information program (Pfeiffer, 1990, 1991a and 1991b). Maryland is the only state that maintains an 
accessible archive of the universe of State Employment Security Agency employment and earnings 
records, dating from 1985 (Middlebrooks and Stevens, 1990). Other states can, in principle, create a 
comparable longitudinal data retrieval capability. 
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The positive features of the State Employment Security Agency quarterly administrative data 

include 

• comprehensive, but not universal, coverage of nonfederal employment, with 
exceptions that are described in the next paragraph; 

• unique employer identification, and full four-digit Standard Industrial Classification 
code detail for each reporting employer; 

• geographic location information for all single-site and some multiple-site employers 
who are required to report; 

• total earnings reported on a quarterly basis with no ceiling cutoff; and, 

• timely access (in some cases within 3 months of the end of a quarter). 

The limitations of the State Employment Security Agency data include the following: 

• There is no occupational information. 

• There is no demographic information about the individual employees; statutory 
restrictions limit a State Employment Security Agency's ability to provide information 
about individuals, because they cannot release information to the public that permits 
identification of either individual employees or employing establishments. 

• Self-employed individuals, commission-only off-site employees, railroad employees, 
federal civilian and military personnel, those employed by some philanthropic and 
religious organizations, and others in special exempt circumstances are not covered. 

• Erroneous, late, and nonreporting by covered employers occurs. 

• Reporting is state-specific, so out-of-state employment is missed. 

• In all but a few states, only total quarterly earnings are reported by an employer for 
each employee, so it is impossible to determine whether an employee worked during 
the entire quarter and whether full- or part-time status is represented. 

• For individuals with multiple wage-records during a quarter, it is impossible to 
determine with certainty whether the employer affiliations were sequential or 
simultaneous, although methods are available to explore this matter. 

• Financial burdens and staff activity priorities limit a State Employment Security 
Agency's ability to respond to recurring requests for information. 

The future availability of these administrative data is likely to remain subject to state-specific 
policies. The controlling statutory authority is at the state level, but the Unemployment Insurance 
Service in the Employment and Training Administration of the U.S. Department of Labor represents 
the federal government's interest in maintaining the fiscal and programmatic integrity of the nation's 
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unemployment compensation system. The Department is developing a policy position with respect to 
the release of these administrative data. Many states have recently completed or have undertaken 
similar reviews of their data release practices (National Commission for Employment Policy, 
forthcoming). 

The State Employment Security Agency administrative data are a promising source of reliable 
employment and earnings outcomes information that can be merged with school-based transcript 
information for National Assessment purposes if appropriate confidentiality requirements and cost 
reimbursement arrangements are made. The inability to extract occupational detail or to derive an 
earnings figure that can be defined in terms of a uniform time unit is balanced by the credibility of 
data collected by a disinterested party whose own administrative uses of the data promote reliability. 
Use of these data would allow the National Assessment to avoid the traditional charge by critics that 
"you only found the ones you wanted to find" in self-reporting by schools of training-related 
placements and wage rates. 

Other potential sources of employment and earnings information that might be added to a 
merged longitudinal unit-record include 

• the U.S. Office of Personnel Management (for federal civilian employees); 

• the U.S. Department of Defense Manpower Data Center (for military personnel); 

• the U.S. Postal Service (for postal employees); and 

• other State Employment Security Agencies (for covered employees in their 
administrative records). 

Florida's Education and Training Placement Information Program uses these additional 
sources of data in its current systems. 

Occupational Information 

There have been state-specific studies (e.g., in Arkansas, Colorado, Florida, and Utah) of the 
feasibility of adding occupational information to the required quarterly report of earnings by 
employers to a State Employment Security Agency. The state agencies resist these efforts "to make a 
Christmas tree" out of their core administrative reporting systems. 

Establishment of Florida's Education and Training Placement Information Program followed 
an unsuccessful attempt to add occupational information to the quarterly wagereport with a 
comprehensive mailed-survey component (FETPIP, 1990). This approach, conducted on a voluntary 
basis and involving half of the more than 40,000 employers of former vocational education students, 
produced an 86 percent overall response rate in its most recent application. Industry-specific versions 
of the Bureau of Labor Statistics* Occupational Employment Survey (OES) codes are provided to 
these employers for their discretionary use in identifying the occupational status of the former 
vocational education students. This represents the current state-of-the-art in collecting useful 
occupational information for descriptive and evaluation purposes. 
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To the author's knowledge, no state has attempted to distinguish among the skill-related, 
required, and sufficiency concepts introduced earlier. It is, therefore, unlikely that the National 
Assessment will be able to report direct evidence about the two more exacting concepts. This is 
unfortunate, given Congress' interest in employment outcomes for special populations. However, 
inferences should be possible from evidence of differences in employment outcomes for former 
students who were enrolled in the same vocational education courses, with appropriate adjustment for 
other influencing forces. 

School-Based Transcript Information 

The State Employment Security Agency administrative data described in the previous section 
exhibit state-level uniformity, substantial comparability across states, and a high level of reliability 
because of their routine use in the administration of the unemployment compensation program. These 
features are less obvious in the world of secondary and postsecondary vocational education. 

Governance of secondary and postsecondary vocational education is highly idiosyncratic. 
State-level management information systems are quite different. The National Assessment's 
evaluation design must be flexible enough to deal with this diversity without losing credibility. The 
credibility of findings will be strengthened if one or more statewide data sources can be identified. 
This will permit the use of appropriate units-of-analysis that are all subject to the same reporting 
requirements. This will also simplify the task of matching school-based and employment and earnings 
records. Credibility will inevitably be threatened by the unknown effect of variability in management 
information system design and use. A state that has a sophisticated statewide management 
information system is likely to be different in important ways from states that have not made this 
investment. This difference may reflect greater legislative support for vocational education, a higher 
degree of concern about vocational education's performance, or a response to management initiatives 
from within the vocational education community. Systemic differences are unlikely to be random. 
Unique combinations of historical circumstances, administrative personalities, and fiscal conditions 
molded the current array of management information systems. 

Reporting systems are routinely modified, which will make it difficult to extract uniform 
information about multiple years of former vocational education students. This will limit the extent of 
which useful statements about the direction and magnitude of change in the employment fortunes of 
former vocational education students can be expected. 

Local Economic Conditions 

Some vocational educators oppose reliance on employment outcomes information. They argue 
that differences among areas at any given time, and within each area over time, make it impossible to 
interpret the outcomes information. The assessment design should respond to this concern by 
including appropriate controls for these differences and changes. This will necessarily be an 
imprecise undertaking, but the effort should be made. State Employment Security Agencies will 
again be an appropriate place to start the search for useable information. 
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The National Assessment's evaluation of employment outcomes for former vocational 
education students will be subject to the following well-known constraints: 

• Multiple sources of measurement error; 

• Severely limited time to compile and analyze pertinent information; 

• Equally limited financial resources to support the inquiry; and, 

• Confidentiality restrictions on the release of unit-record information. 
Two sources of measurement error will be encountered: 

• An inability to include any measure of some forces that are known to 
affect employment outcomes (e.g., motivation, interpersonal skills, 
actual knowledge absorption, and discriminatory actions that affect 
hiring, on-the-job training and promotion opportunities); and, 

• Inaccurate measurement of some factors that are included in the analysis (e.g., 
matching skills acquired through vocational education and the skills required 
in a work setting). 

There have been many previous studies of employment outcomes for former vocational 
education students. The National Assessment can advance what is known about this topic by taking 
full advantage of previously unavailable data sources and data processing capabilities. This can result 
in findings and conclusions that are credible, clear, and useful. 
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Vocational Education and the American Job Market: 
An Employer's Perspective 



Robert L. Martin* 



OVERVIEW 

In reviewing the current vocational system, it has been concluded that vocational education is 
an invaluable form of training that can contribute greatly to federal, state, and local economies. 
However, it appears that the current vocational curriculum is not adequately preparing the majority of 
students for jobs becoming available in the workplace. Technological acceleration has changed the 
nature of jobs in the workplace, leading employers to require higher skill levels than ever before. 
Finding qualified workers to fill those jobs presents an enormous challenge because many entering 
and current employees do not possess the necessary academic and technical skills to keep pace with 
technological change. As a result, America's suffering domestic economy is failing to keep pace with 
international competitors. 

The four sections of this paper 1) discuss the evolution of vocational education in the United 
States, 2) describe vocational education's relation to changes occurring in the American job market, 
3) explain how vocational programs contribute to economic prosperity, and 4) present seven 
conclusions and recommendations and provide background information to support the 
recommendations. 

VOCATIONAL EDUCATION: THEN AND NOW 
Vocational Education: A Brief History 

During the latter part of the 1800s, technical changes generated by the industrial revolution 
and a need to employ and educate a growing influx of immigrants paved the way for a new form of 
schooling which combined on-the-job training with part-time educational instruction. Private 
industries, town councils, and industrial associations began to organize and finance schools offering 
vocational-technical courses designed to train young workers in new skills and supplement their 
knowledge in academic subject areas. Interest in government funding for vocational education 
followed shortly thereafter. 

The push for publicly funded vocational education programs was in part based on the belief 
that public education, if directly linked to the nation's econon-y, could play a vital role in national 
development. Reformers inside and outside the school system became convinced that the central task 
of education was to train youth for jobs and to integrate them into the occupational structure. 
Ultimately, this campaign led to the passage of the Smith-Hughes Act in 1917. In the decades that 
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followed, federal funding led to the development of a vocational education system in every state, with 
growing emphasis on federal, state, and local economic goals. 

The next major reform in vocational education came 46 years later with the passage of the 
Vocational Education Act of 1963 (P.L. 88-210), which introduced training at the postsecondary 
level. The 1963 Act maintained support for occupational training, but also made major public policy 
thrusts by directing funds toward programs that addressed national concerns related to disadvantaged 
and handicapped individuals, to minority populations, and to gender equity. 

Public sentiment toward assisting disadvantaged or otherwise hard-to-employ persons through 
vocational training provisions has grown since 1963. This is evidenced in the Vocational Education 
Amendments of 1968 (P.L. 90-576), which set aside 25 percent of the basic state grant, plus 
additional special funding, for disadvantaged and handicapped students. In 1984, the Carl D. Perkins 
Vocational Education Act (P.L. 98-524) raised these set-asides to 57 percent of the basic grant funds 
and expanded the identified populations to include adults who needed training or retraining, single 
parents, and criminal offenders. 

In brief, the 1984 Perkins Act had two objectives: 1) increasing the quality of vocational 
education instruction by modernizing programs, and 2) expanding access to those programs for all 
persons, particularly those who were disadvantaged, handicapped, or otherwise underserved. The Act 
expired on October 1, 1989. 

On September 25, 1990, President Bush signed into law legislation reauthorizing the 1984 
Act, now referred to as the Carl D. Perkins Vocational and Applied Technology Act (P.O. 101-392). 
The new law abolishes most of the complex set-asides established under the 1984 Act and directs 
virtually all federal money to vocational education programs for poor and disabled students through 
new funding formulas. Services to localities with the greatest concentrations of at-risk populations are 
a primary focus. Other provisions in the new law have significant implications for business and 
work-force training, including 

• Increased focus on basic education skills by targeting funds to programs which 
integrate academic and occupational disciplines and lead to improved worker quality; 

• New tech-prep programs, which offer a combination of secondary and postsecondary 
training leading to an associate degree or 2-year certificate; 

• Increased state discretion in designing and overseeing Perkins programs and 
increased accountability among vocational education service providers; and 

• Business/education partnerships for training, which are encouraged in areas such as 
apprenticeships, new equipment, cash contributions to programs, teacher internships 
and training, and bringing representatives of business and organized labor into the 
classroom. 

It is hoped that these provisions in the new law will help enhance the economic well-being of 
our nation, improve the competence of our work force, and provide meaningful work-related 
educational experiences for all students, particularly those who are at risk. 
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VOCATIONAL EDUCATION AND THE AMERICAN JOB MARKET 



Employment Forecasts for the 1990s 

In many respects, the 1990 Perkins Act reflects significant changes forecasted for the 
American job market during the 1990s. Over the next decade, technology will continue to advance 
and employers will demand higher skill levels than ever before. 

According to the U.S. Department of Labor's Bureau of Labor Statistics (BLS), over SO 
percent of jobs created between 1985 and 2000 will require some education beyond high school. 
Blue- collar or manual-labor positions will continue to decline, from roughly 40 percent of all jobs in 
1970 to 27 percent in 2000. In the interim, demand for white-collar work will escalate. Executive, 
administrative, and professional specialty occupations will comprise 30 percent of all employment 
positions. These jobs require the highest proportion of workers with at least 4 years of college. 
Today, only 22 percent of all occupations require a college degree. 

For the most part, changes in the American job market have been generated by advances in 
technology. Throughout the 1980s, our nation saw great advances in innovative technologies which, 
in turn, led to new products and the creation of new jobs in industries such as communications 
equipment, engineering and scientific instruments, and medical supplies. Developments of this kind 
will continue well into the year 2000. 

Rapid technological change is increasing the complexity of jobs in less technical fields as 
well. Skill requirements are changing dramatically and increasingly require independent judgement as 
well as analytical and interpersonal skills. For example, in manufacturing, machinist occupations are 
evolving quickly from jobs involving simple repetitive motions to jobs requiring technical skills. In 
the service industry, secretaries are becoming information managers, bank tellers are becoming 
financial services portfolio consultants, and insurance claims adjusters are becoming problem analysts. 
In the communications industry, a basic line of voice instruments has turned into an array of systems 
for communicating data and imagery. Even delivery services such as Federal Express now involve a 
computerized tracking system employees are expected to operate and understand. 

In its "Workforce 2000" study, the Hudson Institute, a public policy research center located in 
Indianapolis, examined the skill levels in occupations projected to grow and concluded, "the fastest 
growing jobs will require more language, math, and reasoning skills/ The Hudson Institute data 
show that jobs requiring the three highest skill levels are projected to constitute 41 percent of all jobs 
in the future, compared to only 24 percent requiring those skill levels now. Only 4 percent of new 
jobs will be at the lowest skill level, compared to 9 percent of such jobs today. 

Clearly, workers of the future will have to be able to adjust to changes in job requirements as 
the work environment becomes more technical. Whether business and industry can find qualified 
workers to meet the workplace demands that technology imposes is the challenge that now confronts 
us. 

Educational Outputs and Changing Demographics Offer Little Help 

There is a direct relationship between school systems and the work-force needs of employers. 
Increasingly, the supply of properly trained and educated workers—those able to keep pace with 
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accelerating technological change— is diminishing relative to the need. If current trends in educational 
performance and projected demographic changes are not addressed, business will find it impossible to 
find skilled workers to fill the complex and knowledge-intensive jobs described above. 

America's education system is undoubtedly in a state of deep crisis. Drop-out rates 
nationwide average between 10 and 35 percent and approach or exceed SO percent in many inner-city 
schools. One in five adults is deemed fiinctionally illiterate. 

For students who graduate, the picture is not much brighter. Recent findings in a study by 
the National Assessment of Educational Progress (NAEP) show that reading and writing scores have 
remained virtually unchanged since students in the 4th, 8th, and 1 1th grades were last tested in 1984. 
The average reading scores of all students tested increased by less than one point on a scale of 500. 
The average writing scores of 8th- and llth-graders actually declined. 

The scenario does not improve in the math and science areas. Only one-half of all 17-year- 
olds compute well enough to use decimals and fractions, recognize geometric figures, and solve 
simple equations. About 4 in 10 high school students have a "moderate" understanding of science, 
but only 7 percent have any degree of sophisticated understanding of the subject. U.S. students rank 
poorly in international comparisons as well. A recent study of the math and science proficiency of 
13-year-olds in Korea, Spain, Ireland, the United Kingdom, Canada, and the United States reveals 
that U.S. students rank last in every category except the amount of television watched daily. Another 
study reports that the top 5 percent of today's American math students score at the Japanese average. 

Exacerbating these concerns are "noneducational" problems often associated with public 
schools, such as drugs, truancy, and violence. Teenage pregnancy rates have escalated; every day, 
1,293 teenagers give birth. Every 7 minutes a high school student is arrested for drugs. Roughly 
135,000 students carry a gun to school each day. These problems contribute greatly to growing 
concerns over poor work-force quality. 

Dramatic demographic changes projected for the years ahead will also significantly impact the 
supply of qualified workers. The BLS reports that the civilian labor force will grow by 21 million or 
1.2 percent between 1986 and 2000. This is compared with a 2. 1 percent annual growth over the 
past 15 years. Other indicators are that the total youth population will decline from 43 million in 
1980 to an estimated 34 million in 1996, representing a decrease of 21 percent within this decade. 

BLS data also show that the composition of America's work force will be dramatically altered 
by the year 2000. In today's labor force, roughly 47 percent are white men, 36 percent are white 
women, 10 percent are minorities, and 7 percent are immigrants. Of the net new workers entering 
the labor force in 2000, only 15 percent will be white men. Forty-two percent will be minorities and 
immigrants— groups which traditionally have been the least prepared for jobs becoming available. 
For example, it is estimated that one in every two minority students grows up in poverty, and 
minority dropout rates are extremely high. These trends will severely limit the number of qualified 
entry-level workers joining the labor force in the years that follow. 

Training Needs are Immense 

Over 70 percent of today's workers will still be in the work force 10 years from now. These 
individuals will require training as job positions change and technology advances. 
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The American Society for Training and Development (ASTD) reports that 49.5 million 
workers, or 42 percent of the work force, will need additional training within the next 10 years to 
keep pace with employer skill demands. Sixteen million will need skills and technical training; 5. 5 
million will require executive, managerial, or supervisory training; 11 million will need customer 
service training; and 17 million will require training in basic skills. These figures do not include the 
approximately 37 million workers who will need entry-level training. Attached to these training 
needs is an enormous price tag for employers, who already spend over $30 billion in training, 
retraining, and remedial education each year. 

The fact that many persons seeking entrance into the work force will lack basic and technical 
skills is also discouraging. A recent collaborative study by the U.S. Departments of Labor, 
Education, and Commerce finds significant numbers of employers who cite difficulty recruiting 
qualified workers. Consider the following examples: 

• At Campbell-Mithun Advertising in Minneapolis, the ratio of applicants to those 
qualifying is 20:1 for secretaries and 10:1 for supply and mail clerks. 

• Only 20 percent of persons seeking positions with Motorola can pass a 7th-grade 
English comprehension or Sth-grade math test. 

• The percentage of applicants passing a basic math test for Chemical Bank of New 
York declined from 70 percent in 1983 to 55 percent in 1987. 

In 1990, Towers Perrin, a New York-based management consulting firm, and the Hudson 
Institute released findings from a survey of 645 U.S. companies. All respondents reported recruiting 
difficulties in every employee category, including secretarial, skilled-crafts, technical, and 
professional workers. Table 1 illustrates some of the survey results. 

Table 1 

JOB CATEGORIES EXPERIEKCIBO LIBOR SHORTAGES TODAY 
Percent of Coapmiat Reporting Difficulty Recruiting Bow 

Some Difficulty Great Difficulty 

Secretarial/Clerical 39% 9% 

Skilled Crafts 35 9 

Technical 51 13 

Professional 50 11 

Sales 29 3 

Administrative 31 2 

Supervisory/Management 45 3 

JOB CATEGORIES WHERE LABOR SHORTAGES ARB EXPECTED IB FIVE TEARS 
Percent of Companies Expecting Difficulty in Future Recruiting 

Some Difficulty Great Difficulty 

Secretarial/Clerical 
Skilled Crafts 
Technical 
Professional 
Sales 

Administrative 
Supervisory /Management 

Source: Hudson Institute and Towers Perrin 
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Problems concerning the growing skills gap are not exclusive to large corporations. Small 
businesses, which employ roughly one-half of the nation's private-sector labor force, are directly 
affected. Often touted as the H economic engine" of this country, America's 18 million small firms 
have even fewer workers to choose from during difficult economic times. 

Traditionally, large corporations become more aggressive about hiring when labor markets 
tighten, thus limiting the pool of workers from which small firms can draw. When this occurs, small 
companies either become smaller or forgo expansion. There was a tremendous drop in the growth of 
employment among small firms during 1990. If this trend continues, the prospects for improving our 
economy will be bleak. 

Vocational Education: A Vital National Resource 

Vocational education, which currently serves over 16 million students in approximately 
26,000 institutions nationwide, can be the catalyst in satisfying the skill needs of employers. There 
are numerous benefits to be derived from vocational education— ones that will help lead this nation 
into the 21st century. Vocational training programs instill in young adults skills that are in high 
demand by employers. The National Council on Vocational Education (NCVE) reports that over the 
next decade, 18 of the 20 fastest growing occupations will require vocational education. Vocational 
programs already prepare students for 26 of the 37 occupations the BLS predicts will account for the 
largest number of new jobs by 1995. 

Properly structured, vocational education programs can help ensure that instructors, 
equipment, and curricula provide the labor-market skills needed by employers. Establishing working 
relationships with the private sector can also help vocational education programs remain current. 

ECONOMIC PROSPERITY: HOW VOCATIONAL TRAINING CAN MAKE IT WORK 

Vocational Education = Economic Development 

Our nation's system of vocational education contributes directly to federal, state, and local 
economic development in a number of ways. Vocational education programs increase productivity by 
improving individuals 9 skills and work habits. These programs also support local and regional 
economic growth by providing workers with specific technical skills needed by local employers. 
Vocational education programs can stimulate entrepreneurship. The programs themselves can also be 
viewed as an industry which consumes goods and services and facilitates employment. 

The American public is increasingly recognizing vocational education's link to economic 
expansion. The U.S. Departments of Labor, Education, and Commerce report that 81 percent of 
American citizens see a direct connection between a highly skilled work force and our nation's 
economic position in the world economy. A national public opinion poll conducted by the U.S. 
Chamber of Commerce's Center for Workforce Preparation and Quality Education also found a large 
majority of respondents perceiving a strong linkage between education and economic growth. This 
poll also found that 96 percent of parents with children in the schools are concerned abut educational 
quality. Seventy-five percent agreed that public school systems are failing to prepare students for jobs 
now available in the work force. Sixty percent believe that current workers will not be able to keep 
pace with technological changes in the years ahead. 
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In short, while recognition of vocational education's potential for spurring economic growth is 
well documented, there is some question as to whether our nation's system for occupational training is 
making a sufficient contribution. In fact, there are strong indications that this is not the case— that 
our vocational education system is not structured to meet the growing skill demands of employers. 
The U.S. Department of Education reports that employment projections frequently used to guide 
vocational curricula generally are extrapolations of past employment trends. This means that 
vocational education is ineffective in anticipating "turning points" in employment or training gaps in 
areas of emerging skill needs. 

As noted previously, the lack of basic skills among vocational students cannot be ignored. It 
is reported that only half of all vocational students take algebra, only one-fourth take geometry, and 
only 2 percent take trigonometry or statistics. This is not a promising sign, especially in relation to 
the highly technical, scientific, and quantitative skills employers increasingly desire. Compounding 
the matter is the fact that over 70 percent of persons who will be working in the year 2000 are 
already in the work force. Many of these individuals are functionally illiterate or lack higher-level 
job skills. Eventually, the demand for retraining these workers to meet changing technical needs and 
standards will equal die demand for initial vocational preparation of youth. 

Vocational education contributes to economic expansion in numerous ways. However, it 
appears that vocational programs are not structured to keep pace with economic change. Vocational 
training in basic academic and technical skills is needed for both entry-level and current workers. 

Business Must Enter the Vocational Arena 

Concern about economic development is encouraging business and government leaders to look 
at vocational education as a tool for increased prosperity. When business and vocational education 
leaders work together, they can produce more highly trained new workers and enhance vocational 
education's ability to react to changes in work-force requirements. 

There are many ways for employers and educators to work together to improve students* 
educational and employment experiences and prospects. Variations include efforts designed and 
implemented locally by employers or vocational training institutions, facilitated by local chambers of 
commerce, supported by community-based organizations, or mandated through federal or state 
legislation. No matter what approach is taken, the potential advantages to business and industry, the 
education community, students, and society cannot go unnoticed. 

Federal law already provides for business involvement in external training programs. The Job 
Training Partnership Act (JTPA) (P.L. 97-300) offers employers great latitude in exercising influence 
over local employment and job-training endeavors. The law enables business and education leaders to 
serve as delegates to state-level coordinating councils and to local Private Industry Council (PIC 9 s) 
that put the private sector in full partnership with local government. Provisions in the reauthorized 
Perkins Act call for even more coordination between vocational education and the JTPA. 

CONCLUSIONS AND RECOMMENDATIONS 

Statistics showing a projected decline in the number of qualified entry-level workers joining 
the labor market, in combination with vast production in innovative technologies requiring advanced 
skill levels among employees, point to a skill shortage for America in the years ahead. Many large 
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and small employers already have been affected by poor worker quality; others express fear for the 
future. This report points to seven conclusions about vocational education quality and related work- 
force needs. Those conclusions, along with recommendations for action on the part of the U.S. 
Department of Education's National Assessment of Vocational Education, follow. 

Conclusion 1: For over 100 years, government and industry leaders have recognized 

that vocational education contributes directly to the vitality of federal, 
state, and local economies* The Perkins Act governing vocational 
education reflects strong interest in training at-risk populations and 
attempts to improve work-force quality through new and expanded 
opportunities for business involvement in vocational education* 

Recommendation: Commission an employer survey that examines private-sector 

involvement in Perkins Act services* 

Much effort must be exerted to ensure successful and effective implementation of the 1990 
Perkins Act. Conducting a survey of employers to determine the extent of their involvement in 
programs under the Perkins Act would help serve this purpose. At the least, the survey could 
ascertain whether services under the new Act which specifically address the business community are 
achieving the purpose for which they are intended. Examples of these services include integration of 
academic and occupational training, the new tech-prep programs, increased accountability among local 
vocational education providers, federal grants for exemplary business-education job training 
partnerships, and chamber of commerce representation on local advisory committees and state 
councils. In addition, a survey of this nature would help identify major shortcomings and/or strengths 
in the new law and, in turn, provide direction in terms of future legislative modifications. 

Conclusion 2: Occupations requiring at least 2 years of higher education will increase 

substantially over the next decade, while manual labor positions will 
decline* Technological acceleration will continue to change the nature 
of jobs in the workplace, leading employers to require higher skill level 
than ever before* 

Recommendation: Commission an annual survey that attempts to identify the skills 

needed by corporations of all sizes* 

National, state, and local vocational education systems can better adjust curricula to meet the 
skill needs of employers if they understand the scope of those needs. An employer skill-needs survey 
would promote such an understanding. It would also provide business, education, and government 
leaders with insight into the changing nature of the workplace and help identify differences in skill 
demands among small and large corporations. Comprehensive outreach to disseminate results of the 
survey, together with recommendations for modifications in vocational curricula, would ensure that 
workers eventually possess those skills. Surveys of this nature should also be encouraged at the state 
and local levels. 

Conclusion 3: Despite improvement efforts of the 1980s, American education is in 

dire need of reform. For employers, deficiencies in education mean an 
ill-prepared work force, low productivity, and inability to compete* 
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Concerns about work-force equality are intensified by demographic 
changes looming on the horizon* 



Recommendation: Survey state government leaders to determine the extent to which 

vocational education is incorporated into education reform legislation* 

State-level reform efforts can be abetted by the services vocational education institutions can 
provide. But state legislative and administrative leaders must first be familiar with the potential 
contributions of vocational programs and be given incentives to incorporate those programs into their 
education reform agendas. A national survey of state government leaders would help accomplish this 
task. It would also facilitate vocational education's inclusion in restructuring policies and programs 
targeted for students of all ages. 

Students at risk of dropping out would benefit greatly from enhanced state-level vocational 
programs. Because they often do not adjust well to normal classroom environments, at-risk youth 
may learn better through vocational education programs in which academic training is integrated into 
real-world situations such as the workplace. 

Conclusion 4: Without taking action to improve the basic and technical skills 

foundation of workers in large and small firms, our nation's ability to 
maintain a competitive work force and keep up with technological 
advancements will diminish. 

Recommendation: Investigate the extent to which federal vocational education programs 

serve the training needs of current workers. 

This report has emphasized the importance of enhancing underlying academic skills and 
conceptual understandings to make employees good analysts and problemsolvers. Because roughly 
three-fourths of citizens who will be working in the year 2000 are already in the labor force, these 
individuals might be considered the key to increasing our nation's productivity over the next two 
decades. Their lifelong training needs create yet another demand for the expansion of opportunities 
for worker mobility and transferable skills through postsecondary vocational training. 

A study addressing the degree to which the federal vocational education program serves adult 
workers is needed. Elements of this research could include: 1) the percentage of dollars expended on 
postsecondary vocational training, 2) the percentage of vocational education programs serving adult 
workers, and 3) the level of coordination between vocational education and JTPA services. The 
survey instrument could also be used as a vehicle to collect, evaluate, and disseminate information on 
the best practices in occupational training to interested parties nationwide. 

Long-term benefits could be derived from such a study. Business and industry would be 
reminded of the importance of playing an integral role in governmental training and retraining 
programs. In turn, this would help ensure that training curricula do not become outdated for current 
or future employment needs. 

Conclusion 5: Vocational education is a valuable form of training that effectively 

prepares people for many skilled occupations. 
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Recommendation: Investigate whether state and local programs are meeting the needs of 

vocational education graduates. 

National leadership is needed to dispel common myths about vocational education as providing 
services only to disadvantaged or otherwise hard-to-employ youth and adults. The first step in 
accomplishing this endeavor is to collect comprehensive data on the success rates of state and local 
vocational programs. Success can be measured by determining whether vocational education systems 
are meeting the needs of workers who graduated from vocational training institutions. 

The success rate of state and local vocational education programs should be ascertained 
through a comprehensive employer survey that explores 1) the amount of additional training or 
retraining vocational graduates require, 2) graduates* wage levels and work habits, 3) progress of 
vocational graduates in the job market, and 4) projected employment trends and corresponding skill 
requirements. Qualitative data on employer satisfaction with former vocational students should also 
be included in the survey instrument. 

Survey results could be used to promote the benefits of vocational education. For instance, a 
national trade association could be commissioned to disseminate examples of students graduating from 
vocational programs who have excelled in various occupations and professional fields. In addition to 
providing a focus on the benefits of vocational education, this survey would help federal government 
leaders identify potential weaknesses in the current vocational system, as well as in new programs 
established under the new Perkins Act. 

Conclusion 6: Vocational education contributes to economic expansion in numerous 

ways. However, it appears that vocational programs are not 
structured to keep pace with economic change. Vocational training in 
basic academic and technical skills is needed for both entry-level and 
current workers. 

Recommendation: Conduct research to identify and examine linkages between vocational 

education policy and economic development policy at the state level. 

Vocational education has great potential for encouraging local economic expansion and 
migration of new business and industry into states or localities. However, not all state governments 
have taken action to link vocational education with economic development policy. All too often, the 
process is decentralized and informal, with postsecondary institutions adjusting their curricula around 
local training needs with little or no direction at the state level. 

A survey of state government agencies overseeing vocational education and economic 
development should be conducted to determine whether formal or structured linkages exist at the state 
level. This survey should also investigate the nature of such linkages, especially with respect to the 
employer's role. After survey data are collected, a nationwide exchange of model state policies and 
programs should be undertaken by an appropriate trade association representing business. 

Conclusion 7: By working together, business and vocational education leaders can 

produce more highly trained new workers and enhance vocational 
education's ability to react to changes in work-Force requirements. 
While national opportunities exist through the reauthorized Perkins 
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Act and JTPA mandates, much more can be done at the state and local 
levels. 

Recommendation: Determine the extent to which current business-education partnerships 

involve vocational education* 

Employers need to communicate with vocational institutions to ensure that program offerings 
remain current with technical changes occurring in the workplace. Mutually beneficial partnerships 
with vocational education effectively supplying the skills identified by business and industry should 
therefore be encouraged and strengthened. 

A survey on the extent to which current partnership activities involve vocational education 
should be commissioned. Among other components, the survey should investigate the structure of 
these efforts. Information on whether state and local chambers of commerce play an integral role in 
the partnership would be especially helpful. Once the data are collected, a comprehensive 
dissemination effort promoting the more innovative and successful programs could be undertaken by 
an appropriate federal agency or trade association. Communities could model partnerships that meet 
their local or regional economic conditions. 

CONCLUSION 

The recommendations contained in this report represent an employer's views on what can be 
done to improve the delivery and effectiveness of vocational services and, ultimately, worker quality. 
Because vocational education is a multifaceted industry involving federal, state, and local 
governments; the business community; secondary and postsecondary education institutions; and 
community leaders, recommendations from other perspectives should not be overlooked. If the 
National Assessment were to undertake the various studies described above, vocational education 
might justifiably play an active role in the education reform movement of the 1990s. Given reports of 
an anticipated skill shortage, along with the need to improve America's competitive position in a 
world economy, now is the time to seriously consider the suggestions raised in this report.** 



** The author wishes to acknowledge the significant contributions of Ms. Jill L. Scheldrup, State 
and Local Program Manager for the Center for Workforce Preparation and Quality Education, 
in the preparation of this paper. 
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Part 4: 

General and Special Populations 



Individuals With Special Needs in Vocational Education: 
Considerations for the National Assessment 



L, Allen Phelps* 



INTRODUCTION 

Youth and adults with specialized educational needs will challenge the resources of the U.S. 
educational system in the coming decade. As Pallas, Natriello, and McDili (1989) note, the rapid 
growth of the disadvantaged youth population in the United States over the next decade (one-third of 
the school-age population will be educationally disadvantaged by the year 2020) will require 
considerable additional public resources. As noted in several national reports, failure to respond to 
this severe problem will threaten the nation's economic welfare, as well as our political and social 
stability (Levin, 1985). 

The 1990 Carl D. Perkins Vocational Education Act acknowledges and extends the central 
role vocational-technical education plays in improving economic opportunities for individuals who are 
educationally disadvantaged, including students with disabilities or limited English proficiency and 
students from low-income families. Beyond these groups, the new Act also concentrates resources on 
serving youth who are single, teen-aged parents, individuals in correctional settings, potential 
dropouts, men and women entering nontraditional careers for their sex, displaced homemakers, and 
others who need special assistance in order to succeed in vocational education. Since the landmark 
Vocational Education Act of 1963, a major purpose of federal vocational education legislation has 
been to enhance equitable educational opportunities for the full spectrum of special populations. 

While the funding set-asides for several of the special groups have been removed from the 
1990 Act, several key fiscal provisions emphasize the importance of maintaining needed support 
services and program modifications. In the 1984 Act, provisions for individual assessment, 
counseling, guidance, and transitional services were required for all learners identified as disabled, 
economically or academically disadvantaged, or limited English proficient. In the new Act, priorities 
for fiscal support must be given by local schools and postsecondary institutions to sites or programs 
with "high concentrations of special populations." Funds can also be spent for integrating academics 
and vocational education to ensure that students achieve both academic and occupational 
competencies. In both cases, a recipient district must concentrate federal funds on programs of 
"sufficient size, scope, and quality as to be effective." 

These new policy provisions raise significant questions in an era when special interest group 
concerns are overshadowed by concerns for raising the quality and effectiveness of education for all 
students. As these changes are implemented, the National Assessment will, in part, be a forum for 
debate as to whether or not special interest group concerns have been enhanced or delimited by the 
removal of the funding set-asides and the mandated provision of specific services. 



♦The author is Director of the Vocational Studies Center, School of Education, University of 
Wisconsin-Madison. 
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This paper examines three central issues in the debate: 1) effects of the flexible 
administrative and fiscal policy in the new Perkins Act on access and participation, 2) participation of 
special students in the new, "upskilled" versions of vocational-technical education (e.g., tech-prep and 
integrated academic-vocational education), and 3) the influence of performance standards on the 
equity goals of vocational education. To place these issues in an appropriate context, the paper first 
reviews some demographic trends and examines other federal legislation that has direct implications 
for evaluating vocational education programs and policies affecting special populations. 

SPECIAL POPULATIONS AND THE CHANGING WORK FORCE 

In the 1990 Perkins Act, "special populations" is a broadly constituted initiative. When 
discussing various target groups, significant differences and varied economic, cultural, and social 
issues complicate the policy concerns of individual groups. When school performance data are 
considered, we often find that the within-group differences exceed the between-group differences* 
Thus, it is important in policy-focused assessment to maintain a perspective which recognizes the 
special aspects of disability, poverty, diverse cultural backgrounds, gender differences, and age 
variation. 

Data from the most recent National Assessment (Hayward and Wirt, 1989) and the National 
Longitudinal Transition Study of Special Education Students (NLTS) (Wagner, 1991) indicate that 96 
percent of disabled students took at least one vocational education course during high school. On 
average, disabled students took slightly more vocational coursework than their nondisabled peers: 
Disabled students earned an average of 5.2 vocational credits, compared to an average of 4.0 for 
nondisabled students* However, there are wide variations in the vocational participation rates of 
different subgroups of disabled students, with some groups showing relatively low rates of 
participation. For example, only 49 percent of multiply handicapped, 57 percent of emotionally 
disturbed, and 55 percent of visually impaired students took at least one vocational course. While 
these students represent small segments of the disabled population in comparison with learning 
disabled and mentally retarded students, they too must be served if we are to meet the federal goal of 
providing all handicapped students with appropriate education, including occupational preparation. 

While the enriched forms of vocational education described in the new Act appear to be 
appropriate for all youth and adults if they are to succeed in technology-intensive workplaces, caution 
is urged when examining the effects of these programs upon individuals who come from varied 
cultural backgrounds and have different cognitive abilities. Perhaps the biggest challenge confronting 
vocational educators is providing students from increasingly diverse ability levels with competencies 
and skills that are within their reach intellectually and that enable them to participate fully in 
technology-intensive, teamwork-oriented, and problem-solving workplaces. 

A number of recent reports and studies have documented the growing diversity found among 
the nation's work force entrants* Johnston and Packer (1987) note that as the nation moves toward 
the year 2000, the work force will grow slowly and will become older, more female, and more 
disadvantaged* Nonwhites will comprise 29 percent of the new work force entrants, twice their 
curreot share* Approximately 600,000 legal and illegal immigrants annually are likely to enter the 
United States; two-thirds will enter the labor market. 

Perhaps more importantly, vocational education programs will be confronted with special 
students who have severe and intensive problems. The National Educational Longitudinal Study of 
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1988 (NELS 88) offers a frightening portrayal of the at-risk factors found among students who are 
scheduled to graduate from high school in June 1993 (Hafher, Ingels, Schneider, and Stevenson, 
1990)* It includes the following: 

Forty-seven percent of the class of 1993 have at least one of the following risk indicators (20 
percent have two or more indicators): 

1. Live in a single-parent family 

2. Family income is less than $15,000 

3* Stay at home alone for more than 3 hours 
daily 

4. One or both parents are without a high school diploma 

5. Have a sibling who dropped out of school 

6. Have limited proficiency in English 

Vocational educators will be challenged to serve more intensely disadvantaged students than in 
the previous 25 years of federal interest in special groups. Documenting the efficacy of vocational 
education programs and collaborative efforts with other social and educational programs serving 
students from inner-city, economically depressed areas will be particularly crucial to the continuation 
of federal support for vocational education in future reauthorizations. 

Critics of vocational education have often cited the fact that it has little impact on the dropout 
rate or on providing significant economic benefits for minorities or women. However, the NLTS 
recently has documented more clearly the retention effects of secondary vocational education 
programs for students with disabilities. This study collected comprehensive data on 8,000 special 
education students who were between the ages of 13-21 during the 1985-86 school year and who 
either were or had been in special education programs. Data were collected in 1987 and 1990. (An 
overview of the NLTS is appended.) In a recent report, Wagner (1991) noted: 

Participation in secondary vocational education is one educational intervention that appears to 
hold potential for positive school performance, as well as positive post-school outcomes. 
Across several in-school and post-school outcomes, students enrolled in occupationally 
oriented vocational education were significantly more likely than nonparticipants to register 
positive outcomes, independent of the characteristics of the students who enrolled. Students 
who took occupationally oriented vocational courses had significantly lower absenteeism from 
school and a significantly lower probability of dropping out, when demographic and disability 
differences were controlled. Independent of its effects on students* decisions to drop out, 
having had vocational training was significantly related to a higher likelihood of finding a paid 
job and of attending a postsecondary vocational school in the early years after high school. 

Regrettably, the participation data reveal that only 56 percent of the students with disabilities 
were enrolled in vocational education during their most recent year in high school. 
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Recent Federal Legislation 



In the past year, two significant pieces of federal legislation have been enacted pertaining 
directly to the education and employment of persons with disabilities. Both the Americans with 
Disabilities Act (ADA) of 1990 (P.L. 101-336) and the Individuals with Disabilities Education Act 
(IDEA) (P.L. 101-476) will require vocational-technical education programs to expand their services 
for youth and adults with disabilities. 

The ADA gives greatly expanded civil rights protections to disabled persons in private sector 
employment (by 1994, to include firms with 15 or more employees), all public services, public 
accommodations, transportation, and telecommunications. The ADA definition of a disabled person is 
used in the new Perkins Act, which will expand the types of disabilities needing to be accommodated 
beyond those traditionally served in special education programs. The definition is comparable to the 
one used in vocational rehabilitation programs and includes all persons with physical or mental 
impairments that substantially limit the individual in work and other related life activities. Persons 
with mental illness and heart disease will now need accommodation in vocational-technical programs. 
In addition, the requirements for employers to provide reasonable accommodation in hiring persons 
with disabilities establish important opportunities for vocational education programs to model the 
types of adjustments and adaptations that can be made, thus illustrating the viability of employing 
individuals with different types of disabilities. Implications are also present for expanding the job and 
career placement services provided by vocational-technical programs. Such services need to ensure 
that individuals with disabilities are assisted fully in their search for employment. 

The IDEA redefines the federal assurance of a "free and appropriate education" for every 
disabled child 3-21 years of age. An appropriate education now encompasses transitional services, 
which are to be included in the individualized education programs of students between the ages of 16 
and 21. Transitional services are defined as 

...a coordinated set of activities for a student, designed within an outcome-oriented process, 
which promotes movement from school to post-school activities, including postsecondary 
education, vocational training, integrated employment (including supported employment), 
continuing and adult education, adult services, independent living, or community 
participation. (Sec. 1401) 

With these new provisions, parents and others will see secondary vocational education 
programs as increasingly important for all disabled children prior to their graduation from high 
school. The progress that has been made in steadily increasing the number of students with 
disabilities served in vocational programs must be maintained. 

PARTICIPATION AND ACCESS 

Several significant changes in the Perkins Act are likely to affect the participation of special- 
needs groups. In the reauthorization debates and hearings, critics argued that the removal of the 
matching and set-aside funding provisions would seriously erode the resource base for serving this 
population. More generally, this change was seen by many as another indication from the federal 
government that equity concerns are decreasing in importance, especially in relation to the general 
need for reform in secondary education. Ensuring access to, and effective participation in, a high- 
quality vocational education program continues to be a significant goal of the legislation. However, 
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Ac central assessment issues surrounding participation and access require a clear focus on the 
provisions of the Act that influence the delivery of programs and services. 

Issues and Questions 

The 1984 Act required recipients to provide a series of mandated services (including 
assessment, support services, guidance and counseling, and transition services) to all eligible disabled 
and disadvantaged students. The 1990 Act specifies the same set of services and access requirements, 
and requires that eligible recipients give "assurances" that these services are provided. Since state 
and local evaluation requirements do not include assessment of support services for special 
populations, it will be difficult to determine the extent to which the "paper assurances" are indeed 
effective in sustaining the array of specialized services and accommodations that have been developed 
over the past decade to support the participation of special populations. 

Access to high-quality vocational education programs has been documented as enhancing 
employment and other post-school outcomes. Hayward and Wirt (1989) reported that in comparison 
with non-special population students, disabled and disadvantaged students received nearly twice as 
many of their high school vocational credits from area vocational centers. Area vocational center 
programs tend to be more occupationally specific in focus and have more up-to-date equipment than 
programs in comprehensive high schools. As noted earlier, Wagner (1991) finds that disabled 
students who enroll in occupationally specific vocational education during their last year in school are 
significantly more likely than disabled students who do not participate in vocational education to 
register positive outcomes (i.e., retention in school, employment, and further education). 

Wermuth and Phelps (in press) offer a framework for specifying the components of high- 
quality vocational education programs for special students. This preliminary framework identifies 19 
components of effective programs in five general areas. The components were derived from a 
detailed content analysis of several recent empirical and qualitative studies which examined 
vocational-technical programs serving special population students. In addition to occupationally 
specific instruction, other frequently cited components of highly effective programs were work 
experience, individualized curriculum modifications, cooperative learning strategies, and job 
placement and folio wup services. 

Suggested Inquiries 

Given these dimensions of the access and participation issue, the National Assessment should 
consider several different approaches in the upcoming study. 

The local implementation studies from the earlier NAVE, which involved survey work and 
community-level case studies in selected states, should be considered as a baseline for the current 
assessment. In essence, these studies (or a subset) could be replicated to examine the changes that 
have occurred relative to such variables as: enrollment patterns and rates of participation for special 
groups, support services provided, local interagency relationships with other agencies serving these 
populations, expenditures per special student, post-program outcome data, and perceived problems in 
implementing the new Act. 

Similarly, the National Assessment may want to consider a study which would examine the 
needs assessment data and initial local plans compiled by a random sample of high schools, 
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postsecondary institutions, and other eligible recipients in selected states. Such an inquiry would 
reveal the various strategies used by local administrators responsible for formulating and 
implementating the federal/state program at the local level. 

Developing an understanding of how increased local responsibility for program improvement 
and equity manifested itself in the early stages of the Perkins Act is crucial. The proposed study 
would begin to describe the local interpretations involved in selecting programs that were of sufficient 
size, scope, and quality to be included in the local application. This study could also reveal the plans 
for sustaining provision of support services for special students, depending upon the guidelines given 
to eligible recipients for preparing their local applications. (Much of the ambiguity surrounding this 
issue is attributable to the lack of specificity and clarity in the Act and the lack of regulations prior to 
the first submission date for the state plan. Whether or not the equity provisions apply to all 
vocational-technical education programs operated by an eligible recipient was debated intensely during 
the negotiated rule-making process.) 

Finally, a careful analysis of local applications in selected states would reveal the extent to 
which all schools and postsecondary institutions are receiving federal funds for improving services to 
special groups. The new requirements for minimum grants of $15,000 (secondary) and $50,000 
(postsecondary) may be restricting access to larger communities and to programs for which consortia 
are formed. Special needs populations residing in small towns and rural communities may be 
disproportionately affected. 

The NELS 88 study offers an excellent opportunity to examine the extent of participation by 
special students in high school vocational education programs. In 1990, the NELS cohort was in 10th 
grade; many of the students will be seniors in 1992. If the timelines for releasing these data are 
expedited, it may be possible to use this data base to ascertain how much and what types of vocational 
education were taken by special population students during their early high school years. 
Comparisons to the sophomore cohort of the High School and Beyond/1980 students could be useful 
in determining the trend lines relative to vocational education coursetaking. 

The national longitudinal studies of Chapter I students and the NLTS are other data sources 
that should be examined. These data sources could be useful in establishing trends in access and 
participation, as well as student outcomes derived from programs where various combinations of 
support services were provided at different levels of intensity. 

To date, th e issue of personal value and meaning of vocational -technical education for youth 
and adults with special needs is largely unexplored. The experiences and lives of disabled and 
minority youth are distinctly different in many respects from those of able-bodied whites. 
Understanding how individuals from these groups view the relevance of vocational education is 
important from a policy perspective. There may be cultural and other types of inherent, socially 
ingrained biases that limit participation of certain students in vocational-technical education. 

The National Assessment should consider funding a qualitative study which would examine a 
cross section of 75-150 youth and adults to gain insight regarding their perceptions of the nature of 
work and the requisite educational requirements, the stereotyping and tracking aspects of enrolling in 
vocational education, the role of vocational education in preparing individuals for work, the aspects of 
vocational education that contribute to general education (e.g., family and consumer skills, * 



technological awareness), and other culture-related or disability-related influences. If the sample were 
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divided between youth and adults who had and had not participated in vocational-technical education, 
some interesting comparisons could be made regarding how vocational education is viewed ftom 
various personal-social-cultural perspectives. The findings would be useful in shaping federal policy 
that would reflect the unique considerations and needs of individual groups rather than maintain the 
broad set of policy instruments currently used. 

PARTICIPATION IN UPSKILLED VOCATIONAL EDUCATION 

The 1990 Perkins Act is clearly focused on providing an enhanced set of competencies and 
outcomes. As the preamble to the Act indicates, vocational education programs need to center efforts 
on building a world-class work force. The more advanced and sophisticated skills recommended in 
several recent studies (e.g., Kane et al., 1990 and Carnevale et al., 1990) emphasize the need for all 
vocational-technical education graduates to acquire higher levels of problem-solving and creative- 
thinking skills, scientific and technical knowledge, teamwork skills, and leadership capacities. 

The tech-prep and integration of academic and vocational education elements of the Act have 
drawn significant national attention from both the education and business communities. These new 
thrusts have been described by many as the "new, reformed" version of vocational education. In both 
of these initiatives the clear expectation is that students must acquire core academic knowledge in 
communications, math, and science related to the technical subject area. As the academic rigor of 
vocational-technical programs is enhanced, the matter of providing equitable opportunity for 
individuals with disabilities and disadvantages becomes a paramount concern. 

Issues and Questions 

Traditionally, many adults with special needs have been employed in entry-level, low-skilled, 
and low-wage occupations in the service industry (e.g., food service, building maintenance, and 
household services). Hence, societal expectations often suggest that it is appropriate for special 
students to be prepared solely for these low-wage occupations. Contrary to popular belief, recent 
studies have begun to indicate that special students are capable of succeeding in the full range of 
occupational careers. Hayward and Wirt (1989) report that only 12.6 percent of vocational credits 
earned by disabled students were in the service occupations noted above. Similarly, only 13 percent 
of the vocational credits earned by academically and economically disadvantaged students were in the 
service occupations. Both of these figures compare favorably with the 9 percent of vocational 
education credits earned by nondisabled and advantaged students in these service occupations. 

To ensure that individuals with special needs are given full opportunity to participate in the 
work force, it is crucial that they have access to individually appropriate, demanding vocational- 
technical programs leading toward occupations which provide good earnings potential and 
advancement opportunities. Highly specialized instructional strategies will be needed to ensure that 
students from all ability levels have the opportunity to succeed in vocational education. 

As special-needs students' participation in the full range of occupations increases, societal 
expectations may change. For example, parents of learning disabled, mentally retarded, and 
academically disadvantaged students who become aware of occupational opportunities in fields such as 
fiber optics and waste water technology may change their views of "appropriate" education for their 
children. 
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Suggested Inquiries 



Obviously, studies of state and local implementation which are undertaken by the National 
Assessment should examine the level of participation by special population students. Educational and 
employment outcomes for students in these programs should also be examined to ascertain the extent 
to which outcomes are different for individuals with special needs, such as those who are physically 
disabled or are limited English proficient. This issue is of increasing concern as the current literature 
describing these upskilled programs is being expanded, with little mention of how curriculum 
modifications are made and support services are provided for students with different learning needs. 
Essentially, any efforts by the National Assessment to assess the implementation, quality, and impact 
of these programs should also consider the matter of equitable opportunities. 

The 1990 NLTS data base could be analyzed to examine the 3-year (1987-1990) educational 
and employment experiences of students with disabilities. It would be interesting to track the course 
work and employment of students who have left high school to determine the degree to which 
students are integrating their high school vocational education courses with related employment and 
postsecondary course taking. 

Over the past two decades, the federal government has supported a number of postsecondary 
and higher education programs to serve disabled students (e.g., National Technical Institute for the 
Deaf) and disadvantaged youth (e.g., Upward Bound). A series of papers might be commissioned to 
examine the recent research and evaluation studies from these programs with the purpose of 
identifying specific attributes from successful programs that could be replicated in postsecondary 
vocational-technical education. Such analyses might reveal the extent to which internships, 
experiential learning, career counseling, support groups, tutorial services, and other interventions are 
related to program completion. With the expanded participation of special population students in 
community and technical colleges, it becomes increasingly important to develop policies and programs 
to help ensure that these students complete their programs. 

EFFECTS OF PERFORMANCE STANDARDS ON EQUITY GOALS 

The current national interest in improving accountability in public education will certainly not 
bypass vocational- technical education. However, the "upskilling" of vocational-technical education 
and the establishment of performance measures for use in evaluating programs coincide in this 
legislation to raise a series of crucial questions. 

Issues and Questions 

The most significant issue from a policy standpoint is the "creaming" question: If 
employment and learning-gain outcomes (as required in Section 1 IS) are set sufficiently high to meet 
industry standards, will disincentives be created for serving special population students? If 
performance measures do not encompass equity and access concerns, little attention is likely to be 
given to serving these individuals. Hoachlander (1990) has suggested that, in addition to learning and 
labor-market measures, access indicators be adopted as performance measures for districts or 
programs. Access measures would include enrollment and program completion data which indicate 
the percentage of special population students enrolling and/or completing programs. Operating on 
principles of distributed equity, the percentages could be used to make comparisons with the incidence 
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rates of special population students in the school or community populations or to determine which 
programs limit access for certain groups. 

Evans (1988) has suggested that creaming incentives are reduced when program evaluations 
are based on individual trainee gains in performance. Since full discretion is given to the states and 
their committees of practitioners in determining performance measures, it is likely that conventional 
measures will be adopted initially. Generally, job placement and program completion rates are 
computed at the program rather than the student level. Unless access measures are adopted at the 
program level or value-added measures are used at the student level, it seems likely that all incentives 
for serving special populations in mainstream vocational-technical programs will disappear. 

Suggested Inquiries 

As state boards move toward planning and implementing statewide performance assessment 
systems as mandated in the Act, assessments should be made of attention to equity concerns. To what 
extent do the performance measures adopted by states reflect the intent of the Act to serve 
communities and schools or postsecondary institutions with high concentrations of special populations? 
What approaches are taken by states and local recipients to develop measures which reflect access and 
value-added principles? As the National Assessment examines implementation of performance 
measures and standards, through case studies or by collecting survey data, it is critical that these 
questions be addressed. 

Finally, the National Assessment might support studies that would examine strategies used to 
implement performance assessment and investigate assessment and evaluation approaches used in 
highly effective dropout-prevention programs, such as the California Academies Programs. The 
California Academies Programs have, developed a comprehensive, longitudinal student-level daia base 
which includes information on attendance, achievement, curriculum choices, attitudes toward school 
and work, and work experience, as well as other data describing the performance of at-risk students 
in grades 10-12 who are enrolled in a career magnet program. During the demonstration phase of the 
program, comparison groups of at-risk students from similar schools also were used. 

A number of evaluation studies have been developed to examine the outcomes of the program 
at various locations. From a policy perspective, one of the most useftil assessment studies involved a 
cost-benefit analysis (Stern et al., 1989). After 3 years of program operation, the net savings to the 
taxpayers of California for the first graduating cohort of academy students was estimated to be $1.0 
million to $1.3 million, or $3,500 per student, in social welfare costs. A broader understanding of 
the program evaluation strategies used in effective programs provides an intuitively valid basis for 
developing and using performance measures, and is more appropriate than simply applying the same 
performance measures and standards across all programs and alt students. 
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APPENDIX A 



Overview of the National Longitudinal Transition 
Study of Special Education Students 

As part of the 1983 amendments to the Education of All Handicapped Children Act (EHA), 
the Congress requested that the U.S. Department of Education conduct a national longitudinal study 
of the transition of secondary special education students to determine how they fare in terms of 
education, employment, and independent living. A 5-year study was mandated, which was to include 
youth from ages 13 to 21 who were in special education at the time they were selected and who 
represented all 1 1 federal disability categories. 

In 1984, the Office of Special Education Programs (OSEP) of the U.S. Department of 
Education contracted with SRI International to determine a design, develop and field test data 
collection instruments, and select a study sample. In April 1987, under a separate contract, SRI 
began the National Longitudinal Transition Study of Special Education Studet!!* 'NLTS). 

In the field of research on youth with disabilities, the NLTS is unique in several respects. 
For many years, the research base on youth with disabilities has consisted largely of studies of 
relatively few youth who were in particular disability categories, in a few school districts or a single 
state, or in specific educational placement or treatment programs. It has been very difficult to paint a 
broad picture of students from this fragmented research base. With the NLTS, findings are based on 
a sample that is large and nationally representative. The data presented here were collected in 1987 
for a sample of more than 8,000 youth who represent the national population of secondary special 
education students who were ages 13 to 21 in the 1985-86 school year. The sample permits us to 
estimate with fairly high precision many of the characteristics of youth with disabilities and their 
experiences in adolescence and early adulthood. Further, the sample is nationally representative of 
1985-86 secondary special education students both as a whole and for those in each of the 1 1 federal 
disability categories separately. Therefore, for the first time we know what the transition experiences 
were for youth with mental retardation, for example, and how they differed from those of youth with 
orthopedic impairments or multiple handicaps. 

The NLTS is also unusual in its longitudinal design. The students for whom data were 
gathered in 1987 are being retained in the study so that followup data can be collected about them in 
1990. These followup data will enable the estimation of trends in experiences as youth age. For 
example, we will be able to describe the movement in and out of jobs and in and out of school that 
often characterizes youth in their early adult years. 

Finally, the NLTS is extremely broad in scope, gathering information on a wide range of 
characteristics, experiences, and outcomes of youth with disabilities, including the following: 

• Individual and family characteristics (e.g., demographics, disability-related 
characteristics). 

• Independent functioning (e.g., residential independence, financial independence, 
functional abilities). 
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Social experiences (e.g., belonging to school or community groups, socializing with 
friends). 



• School programs (e.g., courses taken, support services provided, educational 
placements). 

• School characteristics and policies (e.g., type of school attended, policies related to 
mainstreaming, programs available for special education students). 

• School achievement and completion (e.g., grades received, absenteeism, 
dropout/graduation behaviors). 

• Employment characteristics (e.g., rates of employment, job types and duration, 
wages). 

• Postsecondary education participation in vocational schools and 2-year and 4-year 
colleges. 

• Services provided by the school and other sources (e.g., job training, physical 
therapy, counseling). 

• Parental expectations for youth in the areas of education, employment, and 
independence. 

The breadth of scope provides the most comprehensive picture yet available of youth with disabilities 
during adolescence and early adulthood. 

Study Components 

The NLTS has four major components: 

• The Parent/Guardian Survey. In the summer and fall of 1987, parents were 
interviewed by telephone to determine information on family background and 
expectations for the youth in the sample, characteristics of die youth, experiences with 
special services, the youth's educational attainments (including postsecondary 
education), employment experiences, and measures of social integration. Parents 
rather than youth were selected as respondents for the first wave of data collection 
because of die need for family background information and because, with most 
students still being in secondary school and living at home, parents were believed to 
be accurate respondents for the issues addressed. A survey will be conducted in the 
fall of 1990, when youth will be interviewed if they are able to respond. 

• School Record Abstracts. Information has been abstracted from students' school 
records for their most recent year in secondary school (the 1985-86 or 1986-87 school 
year). This information relates to courses taken, grades achieved (if in a graded 
program), placement, related services received from the school, status at die end of 
the year, attendance, IQ, and experiences with minimum competency testing. In the 
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second wave of data collection in 1990, secondary school transcripts will be sought 
for all youth who were in secondary school at any time since the 1986-87 school year. 

• School Program Survey. Schools attended by sample students in the 1986-87 school 
year were surveyed for information on enrollment, staffing, programs and related 
services offered to secondary special education students, policies affecting special 
education programs and students, and community resources for the disabled. A 
similar survey will be conducted in 1991 for youth still in secondary school in the 
1990-91 school year. 

• Explanatory Substudies. Studies involving two subsamples of youth have looked in 
greater depth at 1) students' secondary school programs (the school program 
substudy), 2) the patterns of transition outcomes achieved by youth who were out of 
secondary school (the exiter substudy), and the relationship between school 
experiences and outcomes. Substudies were conducted in 1989 and 1990. 

The NLTS Sample 

The NLTS sample was constructed in two stages. A sample of 450 school districts was 
randomly selected from the universe of approximately 14,000 school districts serving secondary 
(grade 7 or above) special education students, 1 which had been stratified by region of the country, a 
measure of district wealth involving the proportion of students in poverty (Orshansky percentile), and 
student enrollment. Because not enough districts agreed to participate, a replacement sample of 178 
additional districts was selected. More than 80 state-supported special schools serving secondary-age 
deaf, blind, and deaf-blind students were also invited to participate in the study. A total of 303 
school districts and 22 special schools agreed to have their students selected for the study. 

Analysis of the potential bias of the district sample indicated no systematic bias that would 
have an impact on study results when participating districts were compared to nonparticipants on 
several characteristics of the students served, participation in vocational rehabilitation programs, the 
extent of school-based and community resources for the disabled, the configuration of other education 
agencies serving district students, and metropolitan status (see Javitz, 1990 for more information on 
the LEA sample). Bias may exist, of course, on factors for which data were not available for such 
comparisons. 

Students were selected from rosters compiled by districts, which were instructed to include all 
special education students in the 1985-86 school year who were in grades 7 through 12 or whose 
birthdays were in 1972 or before, whether or not they were served within the district or outside the 
district (e.g., in a state-supported residential school). Rosters were stratified into three age groups 



lf The 1983 Quality Education Data, Inc. (QED) data base was used to construct the sampling frame. 
QED is a private nonprofit firm located in Denver, Colorado. Special education cooperatives and other 
special service units were not sampled directly (83% of special education students are served directly by 
school districts; Moore et al., 1988). However, instructions to districts for compiling student rosters 
asked districts to include on their listing any students sent from their district to such cooperatives or 
special service units. Despite these instructions, some districts may have underreported students served 
outside the district. 
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(13 to 15, 16 to 18, over 13) for each of the 11 federal disability categories and youth were randomly 
selected from each age/disability group so that approximately 800 to 1,000 students were selected in 
each disability category (with the exception of deaf-blind, for which fewer than 100 students were 
served in the districts and schools included in the sample). 

In part because of the time lapse between sample selection and data collection, many students 
could not be located at the addresses or telephone numbers provided by the schools. Of the 12,833 
students selected for the sample, about one-third could not be reached by telephone for the parent 
interview. (For more than half of these, addresses and telephone numbers were not provided by the 
schools/districts from which they were sampled.) This relatively high rate of inability to reach 
sample members confirmed the importance of including in the NLTS a substudy of nonrespondents to 
determine whether those who were reached for the telephone interview were a representative sample 
of the population to which the study was intended to generalize. To identify whether bias existed in 
the interview sample, interviewers went to 28 school districts with relatively high nonresponse rates to 
locate and interview in person those who could not be reached by telephone. Of the 554 sought for 
in-person interviews, 442 were found and interviewed, a response rate of 80 percent. A comparison 
of telephone interview respondents with in-person interview respondents showed that the telephone 
sample underrepresented lower-income households. The sample was reweighed to adjust for that 
bias, as described in the next section. 

Of the 10,369 sampled students for whom addresses or telephone numbers were provided by 
schools or districts, some portion of the needed data was collected for 84 percent, the response rates 
for individual components of the study were as follows: 



N 



Response 
Rate 



Parent interview 
School records 
School survey 



7619 
6241 
6672 



71% 

60 

64 
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Recommended Directions: (1) The Effects of Flexibility on 
Delivery of Services to Special Populations and 
(2) Participatory Planning 

Carolyn Maddy-Ber ostein* 



In recent years, the most critical issues concerning vocational programs for special 
populations have centered on access and quality. Indeed, the primary contributing factors in 
reauthorizing the Carl D. Perkins Vocational Education Act of 1984 were findings of the National 
Assessment of Vocational Education (Hay ward and Wirt, 1989) that serious problems exist in those 
two areas. While those issues continue, two additional issues have emerged as a result of the 1990 
Perkins Act: 1) the effects of the new state and local flexibility on delivering services to special 
groups, and 2) participatory planning. 

EXAMINING THE IMPACT OF INCREASED STATE 
AND LOCAL FLEXIBILITY 

The set-aside funds for students with disabilities and students who are disadvantaged contained 
in the Perkins Act of 1984 have been removed by the Carl D. Perkins Vocational and Applied 
Technology Act of 1990. The new legislation requires recipients to channel funds to areas with the 
greatest concentration of special populations and the highest level of economic need. Although 
nationally only 10 percent of the total budget for vocational education comes from Perkins funding, 
the set-aside issue has sparked much debate. Many vocational special-needs professionals predict that 
removal of set-asides from the legislation will result in a reduction of services to special groups. 
Other vocational educators believe this new flexibility will enhance services to the most needy. The 
National Assessment researchers will have to address the issue of which groups are served and which 
are not. A related issue concerns students who traditionally have required more intensive services 
(e.g., students with mental disabilities). Given the recipients* new flexibility to determine who will 
receive what services, what assistance will those who require more intensive services receive? Will 
students who are easier to serve receive the lion's share of assistance? 

In order to monitor the services targeting special populations, Congress stipulated that each 
state's administrators should review the local plans (or a sample of them) of 1) programs for students 
with limited English proficiency, 2) Chapter 1 of Title 1 of the Elementary and Secondary Education 
Act of 1965 (disadvantaged children), and 3) Part B of the Education of the Handicapped Act. 
Evidence is needed to determine the extent and impact of their contribution (Section 111). 

The purpose of the legislation is clearly to serve areas with the greatest concentrations of 
special populations. Yet it would be interesting to determine the impact of the lack of federal funds 



♦The author is Assistant Professor, Department of Vocational and Technical Education, University 
of Illinois and Director, Technical Assistance for Special Populations Program (TASPP), National Center 
for Research in Vocational Education. 
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on vocational services to special populations in less-populated geographic areas (e.g., rural areas) who 
do not qualify for assistance under die new formula. Is there evidence of loss of resources? Are 
states channeling funds to support programs in rural areas? Central to this issue is the amount of 
funding: Are the appropriated funds adequate to accomplish the congressional dictates or has 
Congress mandated an impossible mission? 

Services to Special Populations 

Required services to special populations are extensive at both the state and local levels. State 
plans must include assurances that special populations will have equal access to (Section 118): 

• recruitment, enrollment and placement activities, and 

• the full range of vocational education, including apprenticeship programs. 

State assurances specifically addressing the needs of students with disabilities include 
guarantees that [Section 118(a)]: 

• vocational education will be provided in the least restrictive environment; 

• they have the same rights and protections as students are entitled to under the 
Education for the Handicapped Act (now the Individuals with Disabilities Education 
Act); 

• the student's vocational education plan will be coordinated by representatives of 
special education, vocational rehabilitation, and vocational education; and 

• monitoring will take place to determine if there is consistency between the 
Individualized Educational Plan (IEP) and the program they are receiving. 

States must also provide assurances that students who are disadvantaged and those who have 
limited English proficiency will have access to the "most integrated setting possible" [Section 
118(a)(4)]. 

States must provide an assurance that prior to entering the 9th grade (or prior to whatever 
time vocational education is first taught), students who are members of special populations and their 
parents must receive information on [Section 118(b)(1)]: 

• opportunities in vocational education, 

• eligibility requirements to enroll, 

• courses, 

• available services, 

• employment opportunities, and 
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placement. 



Finally, the states must assure that this information will be offered in a language the students 
and families understand [Section 118(b)(3)]. 

In addition to the state assurance, local recipients of Perkins funds must provide assurances 
that they will [Section 1 18(c)] 

• provide assistance to special populations to help them enter vocational education 
programs; 

• assist in providing students with disabilities the transitional services required under the 
Individuals with Disabilities Education Act (IDEA); 

• provide supplemental services including modification of curriculum, equipment, and 
the classroom; supportive and instructional aids; 

• provide career guidance and counseling services by professional counselors and 
teachers; and 

• provide counseling and instructional services to facilitate their transition to work or 
further education. 

The degree of state and local compliance to these assurances will be a good measure of how 
well the 1990 Perkins Act is implemented. The National Assessment's investiga* will have to 
analyze the extent to which these assurances are being implemented and the impact they are having. 

EXAMINING THE IMPACT OF PARTICIPATORY PLANNING 

A new provision of the 1990 Perkins Act requires states to develop more public participation. 
As in the previous law, the new legislation mandates that states conduct public hearings to provide 
individuals or groups the opportunity to make recommendations concerning their state's plan for 
vocational education. A summary of the public's recommendations with the state board's responses is 
to be submitted with the state plan to the U.S. Department of Education (Section 113). The 
legislation also states: 

The State Board shall: 

1. establish effective procedures, including an expedited appeals procedure, by which 
parents, students, teachers, and area residents concerned will be able to directly 
participate in state and local decisions that influence the character of programs under 
this Act affecting their interests; and 

2. provide technical assistance and design such procedures to ensure that such individuals 
are given access to the information needed to use such procedures. [Section 118(d)] 

The degree to which states comply with these mandates will need to be scrutinized. 
According to recent information gathered by Dr. Carol Kochhar (personal communication, March 
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1991) of the Department of Teacher Preparation and Special Education at George Washington 
University, there are a number of irregularities occurring nationally in the initial steps to implement 
the 1990 Perkins Act. She notes irregularities in hearings due to poor timing of notification about the 
hearings (e.g., notice of the hearing appeared 3 days after the hearing took place; notification 
appeared in obscure places in the newspaper) and insufficient and inaccurate information given the 
public during the hearings. 

To ensure that parents, students, members of local school boards, and representatives of 
institutions of higher education, as well as school personnel, be included in the planning process, 
Congress mandated that representatives of each of these groups be designated as members of the State 
Committees of Practitioners [Section 512(a)]. The National Assessment must determine the 
effectiveness of the procedures established by various state boards to encourage participation of 
parents, students, and others. Evidence of the level and the benefits of participation by each group 
will need to be investigated. 

OTHER CONSIDERATIONS 

In passing the Carl D. Perkins Vocational and Applied Technology Act of 1990, Congress has 
departed from the 1984 Perkins Act in a number of ways. Through the legislation, Congress has 
taken a giant step toward restructuring education while simultaneously bringing vocational education 
into the movement's forefront. Thus, the 1990-1994 National Assessment must not only gather data 
for comparison with findings of the previous NAVE (Wirt, Muraskin, Goodwin, and Meyer, 1989) 
but also must research the effects of several new provisions of the legislation (e.g., tech-prep, 
integration, performance standards) on special groups. 

One case in point is the effect academic and vocational education integration programs have 
on special populations. Grubb (1991) reports several models of integrating vocational and academic 
education. Are some models of integration more beneficial than others? Do some special groups 
benefit more? less? Similarly, do special populations have access to tech-prep programs? What are 
the enrollment rates? Are they receiving supplemental services such as career information and 
guidance to help them enroll in tech-prep programs? 

If Congress is to be guided by the National Assessment findings in 1994 as it was in 1989, it 
is imperative that Congress and the nation obtain the most accurate and comprehensive data possible 
on each of these issues and on each of the targeted populations. 

The Targeted Populations 

Unfortunately, many vocational educators feel that the 1990 Perkins Act has ignored the 
"regular" student by targeting special populations. In reality, the special groups addressed by the 
legislation include much of the nation's general population and certainly a large part of the nation's 
new workers. The legislation defines "special populations" as (Section 521) 

...individuals with handicaps, educationally and economically disadvantaged 
individuals (including foster children), individuals of limited English proficiency, 
individuals who participate in programs designed to eliminate sex bias, and individuals 
in correctional institutions. [Section 521(31)] 
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In addition to the above, the legislation also includes assistance to 



• Criminal offenders (including youth and juvenile offenders); 

• Persons incarcerated in a "correctional institution"; 

• Disadvantaged persons (individuals who have economic or academic disadvantages 
and who require special services and assistance to enable them to succeed in 
vocational education programs); 

• Displaced homemakers (any adult who has worked primarily without remuneration to 
care for the home and family, and for that reason has diminished marketable skills; 
has been dependent on public assistance or the income of a relative but is no longer 
supported by that income; is a parent whose youngest dependent child will become 
ineligible to receive assistance under the program for aid to families with dependent 
children; or is unemployed or underemployed and is experiencing difficulty in 
obtaining any employment or suitable employment, as appropriate); 

• Economically disadvantaged families or individuals; 
« Individuals with handicaps; and 

• Single parents (unmarried or legally separated with custody lor joint custody] of a 
minor child, or pregnant). 

The legislation is obviously geared toward very large segments of our citizenry. Vocational 
educators must not fail to grasp the tremendous need of the nation to prepare all potential members of 
its work force. The perception that "special populations" include only those who have severe mental 
disabilities must be dispelled. The National Assessment's research must emphasize the kinds and 
numbers of people who fit into these special groups to help dissipate these myths. 

RECOMMENDED RESEARCH QUESTIONS 

Many believe the future of vocational education's national funding may hinge on how well the 
1990 Perkins Act is implemented. If the National Assessment's 1994 findings are to be useful to the 
Congress, it is imperative that the questions which drive the assessment's research projects be tied 
directly to the major mandates of the law. The following are recommended questions concerning 1) 
the effects of state and local flexibility in determining services to special populations, and 2) 
increasing participatory planning: 

• To what extent are each of the defined special populations being served? 

• To what extent are areas with greatest concentrations of special populations being 
served? 

• (a) To what extent are special populations being served in areas of greatest economic 
need? (b) To what extent are special populations being served in other geographic 
areas? 
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To what extent are special populations enrolled in integrated vocational and academic 
programs? Are some models of integration of vocational and academic education 
more effective for special populations than others? 



What services are students with special needs receiving to help them enroll in tech 
prep programs? 



(a) How have states differed in their vocational education delivery systems for special 
populations? (b) Are exemplary delivery systems emerging? (c) Are cost-effective 
delivery models emerging? 



Are programs receiving funds of the size, scope, and quality to be effective? 



(a) How well are states complying with the Section 1 18 assurances? (b) How well are 
local recipients complying with the Section 118 assurances? 



What is the level of participation of state-level personnel outside of vocational 
education (i.e., administrators of EH A, Chapter I, and LEP programs) who are 
required to review local plans? 



How well do states announce and conduct hearings on the 1990 Perkins Act? 



What is the level of parent, student, and other non-school personnel participation in 
the State Committees of Practitioners? 



Data Sources 

The former NAVE research (Hayward and Wirt, 1989) centered on access and quality of 
vocational programs for students with disabilities and students who are academically disadvantaged. 
While the current National Assessment will need to compare its findings to those of the earlier study, 
a broader agenda is necessary. Each of the special groups enumerated in the legislation will require 
investigation. The legislation itself is wider in scope than the previous law, and includes new 
provisions such as tech-prep programs and integration of academic and vocational education. 
Furthermore, the removal of the 1984 Perkins Act set-aside funds for people with disabilities and 
those who are disadvantaged has resulted in increased flexibility for states and local recipients in 
delivering services to special populations— another new area for the National Assessment's attention. 

The National Assessment researchers will want to review available data sources before 
launching their own investigations. Among the current research projects being conducted by the 
National Center for Research in Vocational Education (NCRVE) at the University of California, 
Berkeley, is the Tech-Prep for Special Populations project. Through this project, researchers at 
Virginia Tech University (an NCRVE subcontractor) are examining the practices and processes of 
tech-prep programs, especially as they relate to special populations (Hoerner, Clowes, and Impara, 
1991), An NCRVE project in progress at the University of Minnesota (Mortimer, 1991) centers on 
the investigation of vocational education for special populations (i.e., economically disadvantaged). 

An additional source of information is the recently published report, The Benefits of 
Secondary Vocational Education for Young People with Disabilities: Findings from the National 
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Longitudinal Transition Study of Special Education Students (Wagner, 1991). Some other data 
sources National Assessment researchers will want to review are 



• state plans for vocational education; 

• local applications for Perkins fiinds; 

• national, state, and local economic information; 

• information from the state departments of education on the numbers and locations of 
students with disabilities and students who are disadvantaged; 

• transcript data; 

• interviews with appropriate personnel; 

• the former NAVE findings; 

• National Longitudinal Transition Study of Special Education; and 

• ongoing state followup studies of former students enrolled in special education (e.g., 
Iowa, Florida). 

(See appendix for information on additional data sources.) 

Anticipated Problems 

The National Assessment must be completed by January 1, 1994, only a year and a half after 
the law takes effect. This time frame hardly allows enough time to truly measure the effect of the 
law. It is imperative that the National Assessment reports reflect this shortcoming and that Congress 
understand this severe limitation of the findings. The National Assessment researchers must also take 
this factor into account and create designs that show trends and directions. 

Researchers should also be alert to the varying definitions of special populations between 
states and even within some states. For example, some states define a dropout as an individual who 
leaves school during the academic year (generally September-May), but do not include those who do 
not enroll in September. Others count all school leavers in the dropout rate. 

Another difficulty researchers will encounter is the different service delivery systems between 
and within states. A corresponding problem is the various tech-prep and academic/vocational 
education models being implemented. In addition, the mandated performance standards may very 
well result in SO different sets of state standards. 
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Principal Issues Regarding Native Americans to be Addressed by The 
National Assessment of Vocational Education 

Gerald Slater* 



HISTORICAL BACKGROUND 

Any attempt to establish an assessment program for Indian vocational educational programs 
should be informed by at least a cursory understanding of the history of Indian education. This is not 
an easy task, as when one speaks of Indian people, one speaks of over 300 current tribes and an 
estimated 400 or more tribes at the beginning of the 1800s. H.L. Hodgkinson, in Vie Demographics 
of the American Indian, reports that there are currently over 200 languages and dialects spoken in 
Indian communities, out of a total of 400 languages spoken in the United States today. Since 
language is a common characteristic used to identify a distinct culture, this would indicate that while 
Indian communities make up less than 1 percent of the population, they make up 50 percent of the 
cultural diversity. Although the current Native American population is small, it is one of the most 
rapidly increasing populations in the United States today. 

Several national studies have analyzed Indian education. Two of the most noted are The 
Problem of Indian Administration, known as the "Meriam Report of 1928, " and the Senate Report, 
Indian Education: A National Tragedy— A National Challenge, known as the "Kennedy Report of 
1969. w These reports, with minor exceptions, indicate that the U.S. government's policies and efforts 
in Indian education have been a dismal failure. A brief overview of the historv of Native American 
education follows. 

TTie federal government has historically assumed responsibility for educational services to 
Native Americans, beginning with the treaty agreements at the time the reservations were established. 
The first major direction in meeting this responsibility was through agreement v.ith various religious 
sects to provide educational services, including religious instruction, basic literacy, and in some cases, 
basic instruction in husbandry. The motivation for these services was to christianize the "savages." 
The policy divided the nation geographically and allocated the areas to the various religious 
denominations, which established mission schools on the reservations. These educational efforts met 
with very limited success and did little or nothing to help Indian people develop the skills and 
expertise to manage or govern themselves on tribal lands. 

The next major direction of federal policy was a direct attempt at providing education through 
industrial training schools. Although the Bureau of Indian Affairs (BIA) was established in 1835, it 
was not until the 1870s that it played an active role in Indian education. In 1870, the federal 
government designated $100,000 for the operation of federal Industrial Schools, administered through 
the BIA. The first such school, The Carlisle Indian School, was established in 1879 in Pennsylvania. 
The methods used in these institutions were primarily military discipline and promotion of the 
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Christian work ethic. By 1900, about 20 boarding schools had been established. Their educational 
programs were still basic literacy instruction and minimal vocational instruction in husbandry. 

It is important to note that the underlying intention of this policy of relocation was to 
assimilate Native Americans into the dominant culture. Children were placed in boarding schools in 
the early primary grades, and the schools were notorious in their attempts to eradicate any vestiges of 
traditional Indian cultures. The practice of removing very young children from their parents and 
families was inhumane and this policy did not result in assimilation. It did result in tragic tearing and 
unraveling of the social structures of tribes and basic confusion of the identities of the thousands of 
Native American youth who were relocated. The boarding school personnel punished students who 
spoke their native languages and attempted to eradicate practice of traditional religions and the 
cultures* worldview or philosophy. 

The result of this policy was that several generations of Native American people became 
disconnected and alienated from their families and lacked identity and ties to their history or culture. 
The information they were given about their history or culture stated that they were savage, barbaric, 
and inferior to people of European descent. The schools were founded on the premise that Native 
Americans were less intelligent than other people, but good with their hands. Therefore, they were 
directed to the narrowest type of vocational education. The failure of these government policies 
resulted in dependency and despair, and the failure for not achieving self-sufficiency was blamed on 
the victims. Their dependency, despondency, and despair were used as further rationale for criticism 
and as evidence for their lack of intelligence and ability. 

The Meriam Report of 1928 resulted in the next change in the federal government's policies 
toward Indian people. This report pointed out that many tribes had been pushed onto land that whites 
had not wanted and that could not provide a decent living for a trained and experienced white person. 
Regarding the boarding schools, it provided detailed evidence of mismanagement, physical abuse, 
diets that guaranteed slow starvation, and poorly trained school personnel. The report further noted 
that the schools' content stressed only white culture, and that methods of education were worse than 
futile. The report appeared to have a quick and important impact on Indian government policy. Six 
months after its release, The New York Times published an editorial that began, "Is it right to continue 
the policy of trying to de-Indianize the Indians and make white men out of them?" Within 5 years, 
12 boarding schools either closed or became day schools. The curricula began to change and reflect 
Native American interests. In 1932, Franklin Roosevelt appointed John Collier Commissioner of 
Indian Affairs. Collier redefined the government's relationship with Native Americans, stating that, 
"Indian societies, whether ancient, regenerated or created anew, must be given status, responsibility 
and power." In 1934, Congress passed the Indian Reorganization Act. Four main provisions of the 
Act were: 1) a commitment to Indian self-government, 2) consolidation of Indian land holdings, 3) 
economic development for the reservations, and 4) support for cultural pluralism— the restoration of 
cultural traditions. For the first time, after a century of cultural genocide, Indian self-determination 
became the stated government policy. 

Community schools became the thrust of the BIA's new direction in educating Native 
Americans. These new schools were expected to be service centers to the entire community. They 
included bath houses, shops, and libraries that served both adults and children. Native culture was 
included in the curricula. Teachers were provided inservice training, and programs were started to 
train Indian teachers as role models. This new direction in attention to Indian needs was relatively 
short-lived. Lack of government financial resources during the depression and a shift of national 

216 



206 



priorities in World War II resulted in an end to the brief 15-year attempt to solve Indian problems. 
In 1945, John Collier resigned, and the mood of Congress reverted to even harsher attempts at 
assimilation. Several tribes lost their status of sovereignty, legal jurisdictions were transferred to 
states, and many federal services were withdrawn. 

The consequences of this new direction were increased unemployment and severely worsening 
economic and social conditions. In response to this, the federal government implemented a relocation 
program to move unemployed Indians to large urban centers. This policy also failed. Most Indians 
could not tolerate the discrimination, cultural isolation, and separation from extended families, and 
returned home to the reservations, where economic and social conditions had not improved. In 1952, 
the Commissioner of Indian Affairs stated that the ultimate objective of Indian education must be 
"complete integration into the American way of life." So federal policy had again come full circle, to 
assimilation and cultural genocide. 

The next period of change would come with the civil rights movement of the 1960s. Native 
Americans demanded to be heard. Their insistence on an end to assimilation was finally met by 
President Nixon's policy that stated, "right of self-determination of the Indian people will be respected 
and their participation in the planning of their own destiny will actively be encouraged." This new 
attempt at self-determination would be implemented through allowing tribes to contract for the 
provision of federal services. There was no major infusion of federal dollars, rather it was a shift in 
administration of existing funds. Education was still believed to be the primary responsibility of the 
state, and with limited exceptions, continued as before. 

The federal government's policy decision to shift its responsibility for Indian education to the 
states has failed to a great degree. States have excluded tribes from participating in the delivery of 
educational services. Most states explain this refusal on the grounds that reservations are federal 
land, exempt from taxation, and they have no obligation to provide special services to people who are 
not paying for them. This exclusion has been manifested in such things as regulations that prohibit 
Native Americans from voting or holding school board positions. Even after federal law prohibited 
this exclusion, most Indian people are still intimidated by racist attitudes and are reluctant to 
participate. Conflicts over legal jurisdiction and over ownership and control of lands and valuable 
resources such as minerals and water have impeded cooperation in providing meaningful and 
successful education services through state-run schools. 

The newest significant effort in self-determination in education has been the Tribally 
Controlled Community College. The first such college was the Navajo Community College, 
established in 1968 and funded by the federal government through the Navajo Community College 
Act of 1970. In 1978, this Act was expanded as the Tribally Controlled Community College Act. 
Today, there are 24 tribally chartered and operated colleges which share a commitment to cultural 
understanding, preservation, and tribal development. Curricula at these colleges differ according to 
the unique needs of each tribe. These institutions act as a bridge between the Indian world and the 
dominant society. There is a pervading quality of pride in heritage and commitment to community in 
the schools, whose success can be measured by their growth in enrollments— from 1 ,689 students in 
1981 to 4,400 students in 1989. 

Social conditions on most reservations are very bleak and are often likened to those of third 
world nations. Alcoholism is epidemic, and health conditions are among the worst of any group in 
the nation. The cycle of poverty has lasted for hundreds of years and will take years to eradicate. 
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Unemployment rates have historically been the highest in the nation. Historical ups-and-downs that 
have influenced much of the nation's employment have not touched the reservations, where it has 
been all down. The brief interludes of self-determination during the hundreds of years when 
governmental policies have peipetuated dependency and lack of self-direction have not been long 
enough for tribes to create an economic base and a corresponding work force. Therefore, 
unemployment rates range far above the national average of 4-10 percent, up to 80 percent on some 
reservations. There are a few exceptions, such as the Mississippi Choctaw, who have turned the 
corner on extremely high unemployment. They have accomplished this with careful planning, federal 
help through economic development funds, and coordinating training funds, including Perkins monies. 
Although the 1980 Census does not report a total unemployment rate for Native Americans, it does 
supply a comprehensive summary of labor force, income, and poverty characteristics of reservations 
with the largest population of Indians. See attachment A for summary. 

Some state-level data are available. Table 1 indicates the level of unemployment of states 
with rates of over 50 percent. 



Tabic is Statas with Mativa Aaarican UfemployM&t Hatas Ovar 50% 



STATES 


AVAILABLE 
TO WORK 


EMPLOYED 


UNEMPLOYED 


% 

UNEMPLOYED 


South Dakota 


23,108 


6,110 


16,998 


74% 


Wyoming 


2,308 


711 


1,597 


69% 


Mabraska 


1,868 


641 


1,227 


66% 


Minnesota 


7,450 


2,698 


4,752 


64% 


Alaska 


38,107 


14,557 


23,550 


62% 


North Dakota 


9,477 


3,789 


5,688 


60% 


Iowa 


335 


137 


198 


59% 


Washington 


17,935 


7,443 


10,492 


59% 


California 


13,155 


5,556 


7,599 


58% 


Michigan 


4,916 


2,066 


2,850 


58% 


Montana 


15,546 


6,590 


8,956 


58% 


Arisona 


86,061 


37,770 


48,291 


56% 


Maw Msxico 


47,229 


22,135 


25,094 


53% 


Maw York 


5,351 


2,501 


2,850 


53% 
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STATES 


AVAILABLE 
TO WORK 


EMPLOYED 


UNEMPLOYED 


% 

UNEMPLOYED 


South Dakota 


23 r 108 


6,110 


16,998 


74% 


TOTAL 


272,846 


112,704 


160,142 


59% 



Source: "Iodua Service PopuUtiom %md Labor Force Ertuuto," Boreou of IodUa Attain, Jeouary 1M7. 



In sum, if history has taught us anything, it has shown that success in Indian education must 
be based on self-determination and self-governance. Only Indian people can understand and 
implement the kinds of programs that have the proper ingredients of cultural sensitivity and awareness 
of the dominant society. Forced assimilation will not allow for merging traditional Indian cultural 
values with the larger, dominant social and economic structures. Although tribal college vocational 
programs are a special type of tribal program, they sometimes serve as an effective bridge between 
the dominant Anglo world and the traditional tribal worlds because they tend to understand both 
worlds and can speak both languages. Their bridge position is important in helping coordinate 
programs, developing continuity, and helping reduce the high dropout and failure rate among Native 
Americans in the public schools. 

TRIBAL COLLEGE PROGRAMS 

An advantage of tribal college vocational programs is that tribal colleges allow students to 
access normal financial aid programs such as Pell Grants, Supplemental Education Opportunity 
Grants, and Stafford loans. Programs that are not offered through accredited institutions usually 
cannot access these funds. Without access to these and other sources of financial support, adult 
students would not be able to support themselves and their families while they are entered in a 
vocational program. 

The Carnegie Report, "Tribal Colleges," 1989, credits the tribal colleges with reversal of high 
dropout rates among Native American postsecondary students. It attributes this success to the 
following factors: 

"First, the tribal colleges establish a learning environment that encourages 
participation by building self-confidence in students who have come to view failure as 
the norm. 

Second, tribal colleges celebrate and help sustain the rich Native American traditions. 

Third, tribal colleges provide essential services that enrich the communities that 
surround them. 

Fourth, the colleges are often the center for research and scholarship." 

Although there has not been a comprehensive study of success rates at tribal colleges, several 
colleges report success rates of 70-80 percent. A careful analysis of the success of all vocational 
programs serving Native Americans should address the degree to which the factors outlined above 
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exist. It also should examine these programs' efforts to and level of success in reducing or 
eliminating factors that contribute to high dropout rates among Native Americans. 

The tribal colleges draw their basic financial support from funds appropriated for the Tribally 
Controlled Community College Act. The legislation initially designated $4,000 for each fall-time 
equivalent (FTE) Native American student, but the first year's appropriation was $3,000 pe r FI E. 
The current law allows $6,000 per FTE, but the appropriation has decreased to $1,900 per FTE. 
Many of the colleges compete for additional funding, such as the set-aside Perkins funds. In fact, 
many of the tribal colleges that were established over the last 10 years started with Perkins money. 
The current level of Perkins funds to tribal colleges is shown in table 2. 

Table 2 s Distribution of the Kstive lMrican Sat As id* for 19I7-1M1 



TRIBAL COLLEGE TRIBAL OROJWHATIOM 



Year 


Amount 


Niusber of 

Projects 


Amount 


Number of 
Projects 


1987-1988 


$2,749,732. 


12 Projects 


$7,664,620. 


26 Projects 


1988-1989 


$3,341,195. 


11 Projects 


$7,664,620. 


29 Projects 


1989-1990 


$2,955,557. 


13 Projects 


$7,853,433. 


27 Projects 


1990-1991 


$3,371,122. 


11 Projects 


$6,638,825. 


24 Projects 



An important quality of Native American-managed programs is the level of commitment to 
success. With Native American vocational programs, the concept of reform is not an afterthought. It 
is not an exercise in supplanting funds without detection (or continuing the "business as usual" 
attitude) and maintaining the same or a greater level of funding. Rather, reform is seen as necessary 
to avoid the failures of the past and provide solutions for the future. Furthermore, dropping out and 
failing are not acceptable. Indian programs find ways to overcome the problems, and either retain the 
students or recruit them again and again and continue to struggle for success. Anything less is 
unacceptable, because they are family and friends. They are our people, and they are our future, not 
a statistic of past failure. If there is a thought of giving up on a person who is not succeeding, the 
person's grandmother, aunt, or brother will be quick to point out honor is not for those who turn 
their back on others. For Indian people, honor is not given for acquiring great wealth or material 
goods. It is given to those who have a commitment and concern for the well-being of their people. 

The traditions that bind Native American people together, and to their land, are strong. They 
have endured for hundreds of years and through many different policies and administrations. They 
transcend law and acts of Congress, and they have endured through tremendous hardship. They are 
not without change, but the essential elements are still present. The qualities and practices of those 
traditional societies that allowed the Native American people to live in relative harmony with their 
environment for thousands of years— existing without jails or prisons, without mental institutions, and 
without rest homes for the unwanted elderly— have been greatly diminished, but they are still present. 
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ISSUES FOR THE NATIONAL ASSESSMENT 



An increase in the state educational programs' success rate with Native American people is 
imperative, as the majority of Native American students now attend public education provided by the 
state. The distribution is shown in table 3: 



Table 3 s MtmtmfTmdmrml School Attendance Distribution 



Percent in BXA schools 


10% 


Percent in private schools 


5% 


Percent in public schools 


85% 



turn: DiM^UntfAiMflaiMM; K. Hil^U 19* 



Vocational education programs funded by the federal government can play a vital role in 
requiring states to develop advisory committees comprised of Indian people who will ensure cultural 
sensitivity. These committees could also help states avoid past assumptions that the dominant Anglo 
society knows "what's best for Indian people." The committees can help develop and implement 
programs that will match the unique needs of each tribal group and provide the skills and education 
that will best lead to economic self-sufficiency of Indian people. 

To be successful, educational methods and teaching processes must reflect cultural norms and 
values. For example, in most Indian societies cooperation is more important than competition. It is 
rude to stare directly into another person's eyes when speaking to him. It is much more important to 
attend to the need of a family member than it is to attend classes. Facts and information can be 
presented individually, but lessons involved with values are best presented in group activities. 

Wherever possible, educational personnel should be Native American. Role models greatly 
influence students' identification with the program, and students feel more acceptance and a sense of 
belonging. Native American personnel often have a better grasp of Native American students' life 
experience and understand their different cultural values and norms. 

To accurately measure the effectiveness of a tribal vocational education program, it is 
imperative to analyze results in a community-unit context. Tribal programs are very closely linked to 
many aspects of the community. By necessity, the programs share resources and usually cooperate 
closely with most facets of the community. 

The Kennedy Report of 1969 attributed the underrepresentation of Indian people in higher 
education to inadequate academic preparation, teacher and counselor discouragement, financial 
difficulties, and problems with emotional and social adjustment. These problems have not been 
eradicated, and are still primary considerations today. As a result, the dropout rate for Native 
Americans in secondary education far exceeds the national average. A 1988 report from the National 
Center for Education Statistics reports the Native American dropout rate for 1988 at 35. 5 percent, 
while the U.S. dropout rate for this same time was 28.8. In addition to the problem of dropping out, 
there is another problem related to adequate preparation to successfully pursue postsecondary 
education. 
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There are very limited data on the graduation rates of Indian students in postsecondary 
educational institutions. In an article, "A Foot In Each World" (in Black Issues in Higher Education, 
March 29, 1990), Dr. Robert Wells reported a study of 79 colleges with 4 percent or more Native 
Americans. In 1989, only one in four students, or 25 percent, completed the programs they had 
undertaken. The Carnegie Report "Tribal Colleges" reports a failure rate of up to 90 percent for 
Indian students, who leave their reservations to attend postsecondary education and training. 

Native Americans are ambivalent about vocational education— indeed, about most 
education— because of the previous attitude among the dominant population that Indians are less 
intelligent and can only work with their hands. This has resulted in a need for integrated academics 
and applied instruction. Native American educators are well aware of the pitfalls of narrowly focused 
vocational training, as there are already too many Native Americans who are narrowly trained and 
have little or no opportunity for employment. The artificial boundaries that result from narrow 
vocational training go against the Native Americans* basic holistic understanding of the world and 
human's relationship in it. Traditional ways to knowledge and skill are based, to a great degree, on 
merging the humanities and social concerns with the practical aspects of life. Future funding of 
vocational programs for Native Americans should also require a plan for ensuring integration of 
academics and applied instruction. 

Vocational education also must be developed with great consideration for tribal economics. 
Every reservation's economy is unique. It often differs from national trends. As with most 
economies, it is tied to the natural resources within the governmental control. Types of economic 
development will vary greatly from reservation to reservation. Some have potential for developing 
minerals, agriculture, or tourism, and all have an untapped labor force. A universal problem with 
economic development is lack of capital for investment and lack of an infrastructure of supporting 
services such as roads, communications, and general community development that will attract 
industry. In addition, tribal values will greatly dictate how much, and in what ways, resources can or 
will be developed. Tribal beliefs and values will not allow resources development at any cost. Many 
Indian people believe it is better to be poor than to degrade the environment. The Indian religious 
belief that all people are a part of the environment, not the owner of it, precludes development at any 
cost. Therefore, economic development, and the corresponding training, must be done in accordance 
with the tribal values. 

Most tribes have developed an Overall Economic Development Plan (OEDP). These formal 
documents outline the tribe's intentions for economic development, and can serve as blueprints 
indicating the necessary corresponding training needs. Although these documents differ from tribe to 
tribe and vary in their degree of conclusiveness, they can certainly serve as a starting point for 
planning purposes. In addition to their OEDPs, many tribes have conducted manpower surveys to 
help meet the needs of their current local economies and anticipate future personnel and training 
related to projections in their OEDPs. 

There are two types of Perkins funds that impact Indian people: funds awarded to tribes or 
tribal organizations on a competitive basis and the formula-based funds allocated to states to serve 
disadvantaged populations, including Indian people. To ensure that state funds successfully impact on 
Native Americans, the people should be allowed a measure of self-determination, rather than giving 
the states complete freedom to determine what is best for them. It should be noted that most of the 
funds are allocated through public secondary schools and public vocational training schools, which are 
governed and controlled by non-Indian people. The direction and requirements of the new Perkins 
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Act for formula-based funding to the states are not likely to address this concern, as there is no 
required participation of Native Americans. The new legislation does not specifically target Native 
Americans; it targets the "economically disadvantaged." 



Many of the concerns related to formula-based funding to the states do not apply to the Native 
American set-aside funds. Only tribes or tribal organizations qualify to apply for the set-aside funds, 
so tribal involvement is required through this funding source. The set-aside funding is based on 
clearly established criteria. A 65 percent placement of trainee graduates is required. Evaluations are 
required, and failure to complete objectives or comply with the law can and does result in loss of 
funding. The very nature of the competitive process ensures that the best and most innovative 
programs receive funding. 

Data Sources 

There are several major sources of data relating to Indian people. The first is the BIA, which 
maintains demographic information. There are several levels of information sources in the BIA. 
They include the central office, Bureau of Indian Affairs, Department of Interior, Washington, D.C. 
20240, and 12 area offices. Agency superintendent offices are located at each of the 287 federally 
recognized tribes in the United States. The area offices coordinate the affairs of the tribes in the 12 
BIA service areas. The 12 area offices are as follows: 



AREA 



ADDRESS 



Aberdeen 

Albuquerque 

Anadarko 

Billings 

Juneau 

Minneapolis 

Muskogee 

Phoenix 

Portland 

Sacramento 



115 4th Avenue SE 
Aberdeen, SD 57401 

P.O. Box 26567 
615 N. 1st Street 
Albuquerque, NM 87125-6567 

P.O. Box 368 
Anadarko, OK 73005 

316 N. 26th Street 
Billings, MT 59101 

Box 3-8000 
Juneau, AK 99802 

15 S. 5th Street 
Minneapolis, MN 55402 

5th and W. Okmulgee 
Muskogee, OK 74401 

P.O. Box 101 
Phoenix, AZ 85001 

P.O. Box 3785 
1425 NE Irving 
Portland, OR 97208 

2800 Cottage Way 
Sacramento, CA 95825 
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Eastern Area 



1951 Constitution Avenue NW 
Washington, DC 20245 



Navajo Area 



P.O. Box M 

Window Rock, AZ 86515 



Separate and distinct from the BIA office at each of the 287 reservations are the tribal offices. 
These offices are under the authority of the tribal councils and have great variability in size of staff 
and number and types of programs. Most services are either provided directly by the BIA or 
contracted by the tribe from the BIA. So even though services may be administered by a tribe, the 
BIA usually will have records on these programs, although there are some exceptions. These 
exceptions are usually programs that are funded directly by tribal funds or by other non-BIA sources, 
such as the Department of Education, the Department of Labor, or the Department of Health and 
Human Services. Although tribes may have some sources of data, there is usually no clearinghouse 
for information. The best point of contact with the tribe is its executive secretary, who will either 
have the information or know what department to contact. 

Native Americans are one of the most studied groups of people in the United States, and a 
great deal of data exist. The BIA has social information on each tribe. Most tribes (or the BIA) 
handle their welfare systems and have their own employment programs such as JTPA. Many tribes 
have Tribal Work Experience Programs (TWEP), which are synonymous with state work-for-welfare 
programs. Data on these programs and their clients should be available at the tribal level. 

Educational data on Indian people will be less centralized. The BIA will have information on 
education that is administered or funded by them, but it will not have comprehensive data on the 
public school system. As stated previously, 85 percent of Indian elementary and secondary students 
are in public schools. 

The best source of information on vocational education programs funded under the Indian set- 
aside of the Perkins Act will be the programs themselves. Questions of effectiveness can be included * 
in the reporting requirement of each program that is awarded funding. Data on vocational education 
programs administered by tribal colleges could be solicited directly from them. A list of colleges 
follows. 



COLLEGE 



ADDRESS 



Bay Hills Community College 



Route 1 

Brimley, MI 49715 
(906) 248-3354 



Blackfeet Community College 



P.O. Box 819 
Browning, MT 56417 
(406) 338-5411 



Cheyenne River Community College 



P.O. Box 220 

Eagle Butte, SD 57625 

(605) 964-8635 



D-Q University 



P.O. Box 409 
Davis, CA 95617 
(916) 758-0470 
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Dull Knife Memorial College 

Fond Du Lac Community College 

Fort Belknap Community College 

Fort Berthold Community College 

Fort Peck Community College 

Haskell Indian Junior College 

Little Big Horn College 

Little Hoop Community College 

Navajo Community College 
Nebraska Indian Community College 

Northwest Indian College 

Oglala Lakota College 

Lac Courte Oreilles Ojibwa 
Community College 

Salish Kootenai College 
Sinte Gleska College 
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P.O. Box 98 

Lame Deer, MT 59043 

(406) 477-6210 

302 14th Street 
Cloquet, MN 55720 
(218) 879-0800 

P.O. Box 547 

Harlem, MT 59526 

(406) 353-2205, ext. 421 

P.O. Box 490 

New Town, ND 58763 

(701) 627-3665 

P.O. Box 575 
Poplar, MT 59255 
(406) 768-5552 

P.O. Box H-1304 
Lawrence , KS 66044 
(913) 749-8468 

P.O. Box 370 

Crow Agency, MT 59022 

(406) 638-2228 

P.O. Box 269 

Fort Totten, ND 58335 

(701) 766-4415 

Tsaile, AZ 86556 
(602) 724-3311 

P.O. Box 752 
Winnebago, NE 68071 
(402) 878-2414 

2522 Kwina Road 
Bellingham, WA 98226 
(206) 676-2772 

P.O. Box 490 
Kyle, SD 57752 
(605) 455-2321 

R.R. 2, Box 2357 
Hayward, WI 54843 
(715) 634-4790 

Box 117 

Pablo, MT 59855 
(406) 675-4800 

P.O. Box 490 
Rosebud , SD 57570 
(605) 747-2263 
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Siaaeton-Wahpeton community College 



Agency Village 
P.O. Box 689 
Sisseton, SD 57262 
(605) 698-3966 



Standing Rock College 



P.O. Box 450 

Fort Yates, ND 58538 

(701)854-3861 



Stone Child Community College 



Rocky Boy Route, Box 1082 
Box Elder, MT 59521 
(406) 395-4313 



Turtle Mountain Community College 



P.O. Box 340 
Belcourt, ND 58316 
(701) 477-5605 



United Tribes Technical College 



3315 University Drive 
Bismark, ND 58501 
(701) 255-3285 



Problems 



A major problem for tribal programs (and others) will be defining terms. What does 
integrating academics and vocational education mean? How do you define "handicapped?" For that 
matter, what is an Indian? At the time the law to fond tribal colleges was established, 42 different 
definitions of "Indian" were discovered. A list of clear definitions of terms would greatly help 
agencies and individuals respond to surveys or questionnaires. 

Finally, community impact assessment can be done only by a limited sample of focused 
interviews on selected reservations. The federal project officers of Perkins 9 Indian programs could 
identify programs that best meet assessment design criteria. Knowledgeable Native Americans could 
best accomplish this type of interviewing, as they would understand the background and context of the 
tribal programs, and the programs would be more likely to respond with candor. 
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Attachment A 



Summary of 1979 Labor Force, Income, and Poverty Characteristics of American Indiana, Eakimoa, and Aleuts for Select* 



(Data are bated on a sample) 



25 reservations with the largest number 


Percent 


Families with 


Percent of persons 


Median 


of American Indians, 


in 


American 


aged 15 years and 


family 


Eskimos, and Aleuts 


labor 


Indian, Eskimo 


older receiving 


income 




force 


or Aleut house- 


benefits in 1979 






in 1979 


holder or spouse; 










percent with no 










workers 






Total persons 


65.3 


11.9 


41.5 


$9,924 


Blackfeet Reservation, Mont. 


72.2 


6.4 


45.7 


$10,57< 


Colville Reservation, Wash. 


79.7 


7.3 


37.5 


$13,50 


Crow Reservation, Mont. 


70.1 


5.9 


48.6 


$12,231 


Eaitem Cherokee Reservation, N.C. 


73.7 


7.2 


47.7 


$ 9,774 


Flathead Reservation, Mont. 


75.1 


6.5 


47.6 


$11,39: 


Fort Apache Reservation, Ariz. 


64.1 


9.4 


30.8 


$9,273 


Fort Peck Reservation, Mont. 


75.7 


6.0 


51.3 


$10.86- 


Gila River Reservation, Ariz. 


61.5 


15.1 


40.7 


$ 7,95! 


Hopi Reservation, Ariz. 


54.7 


17.6 


31.3 


$ 8,19: 


Laguna Pueblo, N.M. 


70.3 


6.5 


27.6 


$16,75 


Leach Lake Reservation, Minn. 


69.4 


12.6 


52.1 


$ 9,24: 


Misaiaaippi Choclaw Reservation 


68.5 


7.3 


40.6 


$10,53 


Navajo Reservation, Ariz.- N.M.- Utah 


58.3 


16.0 


38.0 


$ 8,39* 


Northern Cheyenne Reservation, Mont. 


66.8 


13.6 


45.9 


$ 9.69< 


Osage Reservation. Okla. 


65.9 


!2.5 


30.1 


$16,09 


Pine Ridge Reservation, S.D. 


61.8 


15.2 


47.7 


$ 7,94! 


Red Lake Reservation, Minn. 


63.2 


15.0 


40.9 


$10,02 


Rosebud Reservstion, S.D. 


65.0 


13.6 


52.2 


$ 3,90] 


San Carlos Reservation, Ariz. 


59.1 


12.3 


43.0 


$ 7,98< 


Standing Rock Reservation, N.D.- S.D. 


65.8 


13.5 


46.6 


$ 8,10 


Tohono O'Odham Reservation, Ariz. 


58.6 


13.5 


50.6 


$7,oa 


Turtle Mountain Reservation, N.D. 


64.8 


17.9 


47.8 


$ 8,83: 


Wind River Reservation, Wyo. 


68.2 


9.3 


31.3 


$10,81 


Yakima Reservation, Wash. 


70.1 


9.6 


44.5 


$1132 


Zuni Pueblo, N.M. 


79.7 


3.1 


46.6 


$10,35 


Source: We, the First Americans, D.L. Johnson, E.L. Paisano, & MJ. Levin (U.S. Dept. of Commerce, Bureau of the Census, December 1988) 
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Special Considerations in Assessing 
Vocational Education in Corrections 



Osa D. Coffey, Ph.D* 



INTRODUCTION 

The Carl D. Perkins Vocational and Applied Technology Act of 1990 places renewed 
emphasis on programs and services to special populations. Under that general heading, "criminal 
offenders" are defined as an eligible subpopulation. The new Act contains many provisions that could 
potentially bring increased federal support for the nation's "correctional" populations. Chief among 
these special provisions are 

1. a 1 percent set-aside (Title II, Part B, Subpart 2, Section 225); 

2. access by corrections to various other initiatives; for example, professional 
development (Section 201), "displaced homemakers" (Section 521), and sex equity 
(Section 222); 

3. access by correctional institutions/agencies classified by the state as an LEA (local 
education agency) 1 to tech-prep funds (Section 343), dropout prevention funds 
(Section 418), Demonstration Projects for the Integration of Vocational and Academic 
Learning (Section 420), and Cooperative Demonstration Programs (Section 420a); and 

4. the creation of an Office of Correctional Education in the U.S. Department of 
Education for the purposes of coordination, provision of technical assistance, and data 
collection. 2 

Although correctional agencies have been able to draw on federal monies for correctional 
vocational programs under previous acts, it was not until 1984 that a special set-aside was created 



*Osa Coffey is currently working as an independent consultant. She is former Superintendent of 
Schools, Department of Correctional Education, Virginia Department of Education, Richmond, Virginia. 
(Author's note: In preparing my assessment design conference presentation and in developing this paper, 
I requested input from a number of colleagues. Many of their ideas have been included in this paper; 
their help is appreciated). 

*Very few state or local correctional agencies are defined as LEAs by their state departments of 
education, although many may be so considered for special purposes. 

2 Since December 1980, such an office has existed intermittently in the Office of Vocational and Adult 
Education (OVAE) in the Department through funding provided by the National Institute of Corrections 
(NIC). This office did not become permanent or fully supported by the U.S. Department of Education 
until mandated by the 1990 Perkins Act. 
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exclusively for criminal offenders. 9 The 1990 act slightly increases the set-aside and provides access 
for corrections to other parts and sections. However, the exclusion from the 1990 Act of set-asides 
for students with disabilities and those who are disadvantaged (traditionally potential funding sources 
for corrections), coupled with the current U.S. Department of Education interpretation that the "hold 
harmless" provision (Section 102(c)) only pertains to the 1 percent set-aside, may reduce rather than 
increase federal support for criminal offenders. 4 For example, should the Virginia Department of 
Education interpret the new law and "hold harmless" provision as limiting the Department of 
Correctional Education to the set-aside alone, the agency would lose about $300,000, compared to the 
previous year. 

The 1 percent set-aside for criminal offenders was controversial from the very start. Criminal 
offenders are no one's political constituency. Feared by the public, these offenders are often 
perceived as undeserving when compared to other special populations. They do, however, constitute 
the highest concentration of educationally disadvantaged and illiterate persons in our nation. A 
disproportionately large percentage of this population is also handicapped. 5 With more than one 
million adults behind bars, at an average cost of $20,000 per inmate per year, the cost to society of 
crime and it consequences amounts to billions of dollars every year. Furthermore, the adult inmate 
population is growing at the rate of 12 percent per year. 

It is crucial to determine whether further investment in the education of criminal offenders 
bears fruit in terms of reduced costs for criminal justice, welfare, and unemployment. To date, little 
is known about the exact use and impact of federal vocational (or other) dollars on criminal offenders. 
There have been no national, uniform data collections, nor have previous national assessments of 
vocational education given more than scant attention to this area. The inclusion of and careful 
attention to this special population in the currently planned National Assessment could be a crucial 
start and could begin to fill an important knowledge gap. 

This paper is intended to assist the team in charge of the mandated National Assessment of 
Vocational Education by suggesting ways in which it could include correctional vocational education 
in the assessment. It will address three points: 

1. What are the principal issues and research questions to be addressed by the 
Assessment in this subject area? 



3 A study made at the request of Congressman John Conyers, Jr. (D-Michigan) by the Congressional 
Research Office determined that when the first 1 percent set-aside was instituted, some states gained 
access to federal vocational dollars for the first time, whereas others lost monies by being restricted to 
the set-aside, having previously received funding from several titles and sections of the Act. 

According to a letter sent to the Secretary of Education and signed by Senators Pell and Kennedy, 
Congress intended the hold harmless provision to include all funding previously received from the Perkins 
Act, not just the set-aside. 

5 It has been estimated that between 25 to 40 percent of incarcerated adults and juveniles meet the 
definition of "handicapped" in P.L. 94-142. The percentage among adult offenders is not known, but 
is believed to be high. 
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2. What are the principal data sources, or, if inadequate, what new data should be 
collected? 

3. What problems should be anticipated in conducting research on this subject? How can 
these problems be mitigated? 

The paper will further address some specific issues raised by Dr. David Boesel, Director of 
the National Assessment: 

1. What do researchers and assessors need to know about correctional vocational 
education? 

2. What is known about the 1 percent set-aside? 

3. What kind of surveys and case studies should researchers undertake in corrections? 

4. What is the participation of special populations in corrections? 

5. What is known about post-program opportunities and outcomes? 

ISSUES AND RESEARCH QUESTIONS 

The Funding Process 

The definitions of "criminal offender" and "correctional institution" provided in the Act are 
extremely broad. 6 Hence, the 1 percent set-aside and other ftmds designated for this special 
population might be spread so thin among various correctional subpopulations and types of institutions 
as to have little impact. They also might be concentrated in one or a few areas, leaving out large 
numbers of potentially eligible persons. Therefore, the first cluster of research questions should be 

1. How did each state distribute the federal dollars among criminal offenders? Did the 
state favor any one subgroup such as probationers, parolees, prison inmates, pre-trial 
detainees, first offenders, or youthful offenders? If any one subgroup was selected above 
others, what was the state's rationale? 

Past experience shows that some states allocated the entire set-aside to one subgroup of 
criminal offenders (e.g., probationers or juveniles). Many states gave the state adult correctional 
agencies all the funds, either for their own use or for further distribution. Although jails house an 



6 It should be noted that although the Perkins Act includes all types of offenders and institutions, 
"corrections" and "offenders" as used in this paper refer primarily to adults and youth incarcerated in 
state correctional facilities. This is due, in part, to the author's immediate experience with these target 
groups. However, offenders who are on probation or parole or in community-based programs usually 
have access to community vocational programs. Such programs should be assessed as mainstream, rather 
than correctional, programs. 
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enormous number of inmates they seem to have been mostly neglected. 7 It is important for the 
National Assessment to try to determine on what basis the correctional allocations were made. 

2. What input, if any, did representatives of various types of correctional institutions have 
in the planning process and/or in decisionmaking related to allocation of funds? 

Although the Act mandates a state council on vocational education, a committee of 
practitioners, and one or more planning committees, the mandate does not specifically require that a 
correctional representative be included on any of these bodies. Until the 1984 Perkins Act, 
corrections had a mandated seat on the state vocational advisory councils. When the mandate was 
eliminated, few correctional representatives were able to keep their seats. The National Assessment 
should study what input corrections had in the crucial processes of developing plans and distributing 
monies. If a lack of meaningful participation is found, Congress may see a need for a designated 
position on these bodies in the future. 

3. Under what various titles and sections of the Act did correctional agencies/criminal 
offenders receive funding? 

As indicated in this paper's introduction, criminal offenders/correctional institutions can 
potentially receive funding from various sections in addition to the 1 percent set-aside, especially in 
the case where a correctional agency is classified as a LEA by the state. Past experience indicates 
that many states have restricted correction to the set-aside, while some states have funded corrections 
more freely under several titles and sections. With the current interpretation of the "hold harmless" 
provision, it becomes easy for states that do not wish to fund corrections beyond the minimum 
required by federal law to limit correctional populations to the set-aside funds. 

4. Is there a significant relationship between a certain type of administrative structure for 
correctional education and the amount and diversity of Perkins monies received? 

The assessors need to understand how correctional education is organized and administered in 
various states. Typically, correctional education programs in adult prisons are administered through a 
program division within a department of corrections (DOC). However, nine states have developed 
"correctional school districts. " One state (Virginia) has a separate state agency for adult and juvenile 
correctional education. Several states have contracted out the administration and provision of 
correctional education to community colleges or local school systems. Some states have adult and 
juvenile correctional education under one umbrella; some have two separate agencies in charge.' 



7 Although it is true that many inmates spend very little time in a jail due to a great backlog of state- 
sentenced inmates, it is not uncommon for felons to spend up to 5 years in a jail, often without access 
to vocational (or academic) programs. 

•For further information, see Osa D. Coffey, "Trends in the Administration of Corrections 
Education," in Wolford, Bruce ed., Correctional Education: Perspectives on Programs for Adult 
Offenders, National Center for Research in Vocational Education, Ohio State University, 1986. 
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In 1981, Richard Carlson reported (based on the Henry David National Institute of Education 
[NDE] National Assessment) that correctional school district states received significantly more funding 
than others. 9 In a 198S study, the author made similar findings but showed that some states with the 
traditional DOC model did just as well. 

Since the new Act clearly distinguishes between correctional agencies with state-acknowledged 
LEA status and others, the National Assessment should determine what difference the LEA status has 
on funding patterns. 10 

5. Did criminal offenders receive more or less funding under the 1990 Act than under the 
previous one? 

In 1990, the driving force behind the 1 percent set-aside for corrections (as in 1984) was 
Senator Claiborne Pell (D-Rhode Island). It is clear from his correspondence with the Secretary of 
Education (see footnote 4) and other sources that his intention was to increase the funding base for 
correctional education through the set-aside while allowing corrections access to other parts of Act. 
He was acting in the belief that educational programs for adult and juvenile offenders will lead to 
increased chances for rehabilitation, post-release employment, and reduced recidivism. The 
Department of Education's narrow interpretation of the "hold harmless" provision could lead to 
results contrary to the congressional intent. The National Assessment should look into this. 11 

6. What proportion of each state's total spending for vocational education for criminal 
offenders did the Perkins Act monies constitute? 

It is important to determine to what degree states depend on federal funding for vocational 
education for offenders. In a study of this subject conducted by the author in the spring of 1988, 
states reported that the Perkins Act constituted from 0 to 80 percent of their total vocational funds 
(see study referenced in footnote 3). 

7. What was the criminal offender per capita expenditure in each state? 

This question could be ftirther subdivided or focused exclusively on a few correctional 
subpopulations; for example, adult prison inmates, juveniles in state correctional facilities, 
probationers, etc. This is an important question because both crime and incarceration rates fluctuate a 
great deal among the 50 states. Generally speaking, the South has the highest incarceration rates, the 
longest sentences, and the lowest parole and probation rates. The National Assessment's probe in this 



*See Richard Carlson Vocational Education in Prison Setting. Report to the U.S. Department of 
Education, 1981. 

1( TTie Virginia Department of Education will be designated an LEA by state statute, effective July 1, 
1991. This is probably the first time a correctional education agency has become an LEA by statute 
rather than by state "consideration* 1 or regulation. 

n The Conyers study cited in footnote 3 revealed that the 1 percent set-aside increased the funding 
base for correctional education generally, although some states experienced a reduction in Perkins funding 
(as much as $375,000). 
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area could lead to recommendations for a change in the formula for funding correctional education 
that would take into account not only education and economic data, but also criminal justice data. 

Uses of Funding for Criminal Offenders 

The second broad area the National Assessment should explore is how Perkins Act funding is 
being used in corrections. This is the area where corrections could most readily bepSrt of 
"mainstream" assessments. In most correctional settings—especially adult and juvenile secure state 
facilities— the needs to be met by the federal funds are the same as to those in community programs; 
that is, program and curriculum improvement, staff development, equipment upgrade and expansion, 
academic/vocational linkages, equity for females, and special programs and adaptations for 
handicapped students and those with limited English ability. 

During the March 1991 assessment design planning conference in Washington, several 
speakers expressed the opinion that it might be difficult to track spending. It is this author's belief 
that tracking can be done in the correctional area and that most recipients in corrections already keep 
this information. In the Virginia Department of Correctional Education, uses of all federal dollars are 
tracked separately from state general funds, making it possible to account for exactly how each 
Perkins dollar is spent. The problem the National Assessment will experience is that this information 
might not be readily available centrally; that is, in each state education agency (SEA) which 
administered the flow-through funds, and therefore might have to be collected from each local 
recipient. The National Assessment should consider using a sample for this aspect of the evaluation; 
for example, selecting the 10 to 20 correctional agencies in the United States that received the highest 
Perkins allocations. 

Measuring Outcomes in Corrections 

Measuring the outcomes of Perkins Act expenditures in corrections will be by far the most 
difficult task for the National Assessment. However, if it can be done, it will be the best contribution 
the assessment can make to correctional education nationwide. Well-designed, scientific, followup 
studies are virtually nonexistent in this area, yet every legislator on both the state and federal levels 
wants to know if it is worthwhile to invest education dollars in persons who have broken the law. 
The critical question is not "What is in it for the offender 0 " but "What is in it for the taxpayer*!" 

Measuring outcomes is costly and difficult under any conditions. With criminal offenders the 
problems are compounded. In terms of outcome measures, criminal offender programs should be 
treated separately from community-based programs for other students. Furthermore, the only valid 
comparisons will be among offender programs. This is not to imply that criminal offender vocational 
programs should not or could not be of the same quality as other programs. What must be stressed is 
the fact that factors unique to, and often due to the correctional context will affect correctional 
outcomes. Specifically, there are four elements the National Assessment researchers must understand 
and take into account in designing and implementing this aspect of the total assessment: 1) the nature 
of the "typical" criminal offender; 2) the particular constraints of secure correctional environments 
(i.e., prisons, jails, and juvenile detention and correctional centers); 3) the impact of a criminal 
record on employment; and 4) the confusion of correctional outcome measures with vocational 
education outcome measures. 
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According to statistics from the U.S. Department of Justice, the "typical" offender, male or 
female, adult or juvenile, is multiply handicapped and/or educationally disadvantaged. Among adult 
offenders, 80 percent are school dropouts, with an illiteracy rate of 40 to 60 percent. At least 30 
percent have serious drug abuse problems, and another 30 percent have alcohol abuse records. At 
least 40 percent were unemployed at the time of arrest; most of those who worked had less than 
subsistence-level incomes. Estimates show that learning disabilities and other handicapping conditions 
are several times as prevalent among offenders as among the general population. The greatest 
handicap of all is likely to be the criminal record that will follow the adult offender throughout bis or 
her life. The juvenile offender profile is not very different from the adult profile. Typically, juvenile 
offenders are several grade levels below their age group. Increasingly, drugs, alcohol, and violence 
are part of the juvenile offender's history. 

The secure correctional environment usually creates severe constraints on programs. Due to 
overcrowding, fiscal constraints, and the higher priorities usually set on security, correctional 
programs often lack adequate space and equipment for educational programs. Frequent lock-downs, 
tool-control policies, institutional counts, and other factors take large bites out of the time available 
for classes. Frequent— and often sudden— transfers impact on program continuity and completions. 
Uncertainty about parole or release makes it hard to time programs so that completions are possible 
and in proximity to release and possible outside employment. The criminal record is also a hindrance 
to employment. There are virtually hundreds of legal and licensing restrictions that bar ex-offenders 
from a number of employment opportunities. Furthermore, employer prejudice, although hard to 
assess, is a reality. Therefore, regardless of the quality of the vocational programs, ex-offenders are 
less likely to be successful in finding employment than nonoffenders and are likely to spend more 
time finding employment and to settle for lesser-skilled and lower-paid jobs. These factors clearly 
indicate that the assessors cannot compare correctional populations with general populations in terms 
of outcome measures involving time spent in a course, completion, or time between completion and 
employment. 

Finally, a note of caution must be struck in terms of reduced recidivism as an outcome 
measure. Obviously, correctional education is often funded in the hope that upgraded basic skills, 
coupled with a marketable vocational trade, will lead to offenders' speedy employment after release at 
a wage which will permit post-release success and reduce recidivism. However, this view is 
somewhat naive since not all crime is caused by unemployment, poverty, or poor education. Despite 
intense studies in the 1970s of the relationship between unemployment and crime, no direct cause and 
effect has been defined. Several years ago, the California Youth Authority studied youth released 
from correctional facilities. They found 10 separate variables impacting on post-release success or 
failure. Only two of these variables were part of the institutional experience: education and treatment 
programs. Again, one must be careful not to attribute success or failure to one variable when many 
determinants are involved in crime and repeat crime. This is not to imply that recidivism should not 
be considered. Obviously, both state and federal governments invest in correctional education 
expecting a good return on the money. Cost savings can be derived from a number of factors related 
to vocational training for criminal offenders other than reduced recidivism, although that is the biggest 
factor. 
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Based on the above discussion, the following assessment recommendations are offered: 

1. Design an outcome assessment specifically for like correctional programs to determine 
what is more or less effective in terms of vocational programing for criminal 
offenders (rather than comparing offender programs with community programs). 

2. Use impact on recidivism as one outcome factor, not as the factor. Earlier release, 
greater productivity during incarceration (in prison industry or construction/ 
maintenance), a shorter period before employment after release, longer job retention, 
longer periods off welfare, etc., are factors that save taxpayers money in the long run, 
even if repeat crime eventually occurs. To the extent that these factors are present 
and can be directly related to Perkins Act programs, they are positive measures. 

3. In order to collect credible followup data, the National Assessment should invite 
correctional education practitioners to help design the evaluation and select programs 
to participate in the study. The Assessment should also provide some funding to 
participating agencies to help them carry out the study. 

The National Assessment could be in a position to help provide answers to the correctional 
education $64,000 dollar question: "What vocational programs have the highest success with what 
types of offenders as measured by post-release employment, economic status, and reduced 
recidivism?" 

DATA SOURCES 

Systematically collected data on correctional education— including correctional vocational 
education— do not exist. Federal funding for correction education has not traditionally been reported 
separately. In the past, an occasional study (e.g., the one requested by Congressman John Conyers 
and conducted by die Congressional Research Office and another undertaken by the Correctional 
Education Association) has attempted to fill part of this void. At best, some of these studies have 
determined the amount of federal funds states have received at certain times. No data exist that show 
how the funds were spent, with what results, or with what fluctuations over time. For the first time, 
there is currently an initiative under way to try to develop a federal, ongoing, systematic collection of 
correctional education statistics through the U.S. Department of Education. The Correctional 
Education Data Project (CEDP) has developed an instrument for data collection. It has yet to be 
piloted, and no actual implementation can be expected for at least another year. Because of the 
timing, the National Assessment initiative could be coordinated with the CEDP; it might not be too 
late to build the basic National Assessment questions into the draft instrument. This could be 
coordinated through the recently established Office of Correctional Education in the Office of 
Vocational and Adult Education. 

The process and funding data described in the first sections of this paper could be collected 
through the SEAs, possibly with some followup surveys of correctional education recipients. Precise 
expenditure data (i.e., exactly how recipients used the federal funds as opposed to other monies) 
could be collected through recipient surveys if they are not available in the SEAs. 

Outcome data will be the most difficult to collect. The author recommends that an instrument 
be developed with the assistance of practitioners. It should be implemented on a selected basis, as 
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Outcome data will be the most difficult to collect. The author recommends that an instrument 
be developed with the assistance of practitioners. It should be implemented on a selected basis, as 
more will be learned with a solid followup study in a few places than by attempting to collect the data 
on a broad scale. The sample should include correctional vocational programs in a variety of settings 
and under various types of administrative structures, where followup can be done through existing 
systems such as parole and probation data systems, the Offender Based Classification Information 
System (OBCIS), and state employment agency networks. To select the specific sites, programs, and 
correctional education contacts, the National Assessment should solicit assistance from the Office of 
Correctional Education in the Department of Education, the Correctional Education Association 
(CEA), and the Association of State and federal Directors of Correctional Education. Legal 
assistance should also be sought to ensure that all legal provisions for offenders' rights to privacy and 
confidentiality of offender records are followed. This is particularly critical in the case of juvenile 
offenders. 

CONCLUSION 

In the past, national assessments of vocational education have paid scant attention to 
correctional education programs. Correctional educators have not been invited to participate. Few 
programs for criminal offenders were scrutinized. As a result, Congress has had little information on 
which to base its legislative decisions. Since the NIE study in the late 1970s, the correctional 
population has grown by about 135 percent, and has become far too big to be ignored. It is critical 
that we determine how federal funds are best allocated and utilized to maximize their impact on the 
educational needs of criminal offenders. 

The correctional community looks forward to participating in the National Assessment effort 
and hopes that the federal government will scrutinize correctional vocational programs on a par with 
community programs for nondelinquents. By necessity, the National Assessment's primary mandate 
is to provide future Congresses the information needed to make appropriate decisions related to 
amendments or reauthorizations of the Perkins Act. However, the Assessment could also provide 
important information for state and local administrators and program managers to use in improving 
program delivery to students. 

Among other things, the National Assessment is charged to provide both the Congress and the 
U.S. taxpayers with information needed to answer two critical questions: Can society afford to 
make a costly investment in vocational education for criminal offenders? Can society afford not 
to do so? 
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Teacher Preparation, Qualifications, and Demand 

Curtis R. Finch* 



OVERVIEW 

Beginning in the early 1980s, a number of studies and reports related to public education have 
caught the public's attention. A Place Called School (Goodlad, 1984), A Nation at Risk (National 
Commission on Excellence in Education, 1983), and High School (Boyer, 1983) are examples of 
reports that called for significant changes in education. Focusing more specifically on vocational 
education reform were documents such as The Unfinished Agenda (National Commission on 
Secondary Vocational Education, 1985), and Investing in Our Children (Committee on Economic 
Development, 1985). 

Recommendations from these and other reports revealed a number of areas that demanded 
immediate attention. These included: 

1 . Vocational and academic educators must be equal partners in the education process, 
must complement each other, and must actually work together. 

2. The basics are important. Basic education standards must be met by all students. 

3. Basic skills must be made more relevant to each student. Basics should be integrated 
around each student's present and future life-role. 

4. Each student should receive academic and vocational education. 

5. The curriculum should focus on developing the whole student, not only his or her 
basic life- and work-skills. 

6. Business, industry, and the community should have a more active role in the 
educational enterprise. (Finch, 1990) 

In response to reform recommendations, a number of schools across the country have 
implemented meaningful changes. These changes include increasing graduation requirements, placing 
emphasis on integrated academic and vocational education, and improving linkages between secondary 
and postsecondary institutions. 



*Curtis Finch is Professor and Director, Virginia Polytechnic Institute and State University Office, 
National Center for Research in Vocational Education. The remarks presented in this paper are entirely 
those of the author and should not be attributed in any manner to the National Center for Research in 
Vocational Education, University of California, Berkeley, or the U.S. Department of Education. 
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Following closely in the wake of studies related to education reform were numerous 
examinations of teacher education. For example, Tomorrow's Teachers (Holmes Group, 1986) and A 
National Prepared: Teachers for the 21st Century (Carnegie Forum on Education and the Economy, 
1986) recommended that a bachelor's degree in the arts and sciences serve as a prerequisite for the 
professional study of teaching. (Additional teacher preparation recommendations are documented in 
Finch, 1987; Hughes, 1987; Lee, 1987; and Smith, 1987) Focusing more directly on vocational 
teacher education, The Unfinished Agenda recommendations indicated that vocational education 
teachers should attain the same level of education as their academic teacher counterparts. Concern 
was also expressed about the need to update teachers' technical skills and to provide better means by 
which talented individuals could be attracted to and retained in the teaching profession. 

The overall impact of teacher education reform has been great. Some universities have 
decided to abolish their undergraduate teacher education programs. Other universities have decided to 
eliminate all teacher education programs from their offerings (Diegmueller, 1991). In sum, the 
reform movement has had significant and lasting impact on teacher preparation. 

In addition to the impact of educational reform on teacher preparation, teachers* professional 
roles seem to be evolving at a more rapid rate than in the past. For example, academic and 
vocational teachers are being asked to work together in the integration of subject matter content and 
delivery. Teachers at the secondary and postsecondary levels are expected to work together in 
establishing and maintaining tech-prep programs. These changing roles, in turn, have many 
implications for teacher preparation and qualifications. Clearly, the teaching environment— as well as 
teaching content and structure— is changing, and teachers must be responsive to these changes. 

Building on the educational reform movement and teachers' evolving roles, this paper 
examines three important areas related to assessment: 1) teacher preparation, 2) teacher 
qualifications, and 3) teacher demand, including the extent to which demand is being met. These 
three areas are closely intertwined and thus should not be considered completely independently. It 
also should be noted that although relevant information has been drawn from general teacher 
education, the focus is more specifically on vocational education teachers and teaching. 

ISSUES 

Assessment issues logically flow from provisions contained in the 1990 Perkins Act. 
Assessment of teacher preparation, qualifications, and demand must be consistent with and build upon 
these provisions. Issues also flow from the current status of teachers and how well these 
professionals are being prepared for their future roles. A final set of issues is contained in 
documented research priorities related to vocational education personnel development (Lynch, 1987; 
Griggs, 1990). 

Preparation Issues 

What is the optimum mix of preparation studies and experiences for vocational teachers? 
Concern about preparation of teachers in general and vocational teachers in particular is well 
documented. For example, Beyer, Feinberg, Pagano, and Whitson (1989) question the divisions 
between educational and liberal studies many teacher education reformers seem to have taken for 
granted. Lynch and Griggs (1989) also take issue with teacher preparation. They question the notion 
that a degree in arts and sciences (or in subject matter that is most appropriate for vocational 
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education) will ensure success in teaching a particular vocational subject. Researchers would thus do 
well to focus on the various ways vocational teachers are prepared and examine the existing links 
between preparation and actual performance in the teacher role. Unfortunately, this may not be 
feasible during the current assessment cycle. At this time, a realistic assessment goal would be to 
establish an information base that would enable future assessments to focus on links between 
preparation and performance. 

What specific preparation must be provided to prepare vocational teachers for their new 
roles in the education enterprise? Both academic and vocational teachers are or soon will be faced 
with the task of integrating academic and vocational education in their schools. Teachers will also 
participate in establishing tech-prep programs that link secondary and postsecondary institutions to 
better meet students' needs. These changes as well as others have the potential to place new 
professional requirements on teachers. Examples of such requirements might include working as 
members of professional integration teams and coordinating educational content and delivery with 
counterparts in other departments and institutions. Although the exact nature of these requirements is 
not yet known, initiation of new curricular directions in the schools can provide researchers with a 
unique opportunity to examine how and to what degree teachers assume these new roles. 

To what extent are vocational teachers prepared to work with special-needs students? 

This question is often asked and is typically answered by the comment "it depends." Teacher 
preparation programs and certification requirements vary from state to state and once teachers are on 
the job they may encounter students with a wide range of special needs. In the schools, teachers may 
receive support from a great number of specialists or may be on their own. As more and more 
special-needs students are mainstreamed in vocational education classes, it is important to know if 
teachers are capable of helping these young people and adults achieve their maximum potential. 

Are those who currently teach continuing to develop both technically and professionally? 

Continuing to develop is as important to vocational teachers as it is to other professionals. Today's 
professional and technical knowledge and skills may be obsolete unless they are supplemented by new 
information and practice. This need extends beyond full-time teachers to part-time teachers, who are 
becoming an increasingly larger percentage of the faculty members in postsecondary institutions. 

How do alternate paths into the vocational teaching profession compare to traditional 
paths in terms of teacher readiness to teach? It should come as no surprise that about 70 percent 
of the trade and industrial teachers do not have baccalaureate degrees. Many of these nondegreed 
teachers have entered the teaching field with little more than a high school diploma and a vita full of 
rich employment experiences in an occupational field. Others enter the vocational education arena 
after receiving baccalaureate degrees in a variety of fields and working in areas such as business, 
marketing, and agriculture (Lynch, 1990). The flow of persons with such diverse backgrounds and 
preparation into vocational education teaching raises a host of questions about readiness to teach. 
This is particularly true at a time when teachers are being expected to integrate academic and 
vocational education and to establish tech-prep programs. 
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Qualifications Issues 

To what extent do teacher qualifications differ from preparation? Even though persons 
may have completed teacher education programs, it does not mean they will be competent teachers. 
Teachers enter the profession at stages ranging from advanced learners to seasoned professionals, and 
many must learn on the job what they did not learn in universities or while working in technical fields 
(Heath-Camp and Camp, 1991). Therefore, factors such as quality and relevance of teacher education 
programs and backgrounds of persons entering the teaching field create the potential for a great deal 
of variability in vocational teacher qualifications. This potential variability is clearly an area that 
warrants examination. However, the current assessment team may only be able to document the 
qualifications teachers currently possess (e.g., participation in organized presence internships versus 
being placed in front of a class without professional preparation). 

Does teacher preparation contribute to success in new teacher roles? Although tech-prep 
and integration of academic and vocational education are just beginning to be implemented in the 
schools, the next several years will see much growth in these areas. Thousands of teachers probably 
will be engaged in implementing new and refined programs. As these changes occur, it is an 
opportune time to examine how teachers accept new roles and the extent to which this acceptance is a 
function of preparation. 

To what extent is occupational experience an asset or a liability to vocational teaching? 
The debate over how much occupational experience vocational teachers should have has gone on for 
decades. However, structural changes in vocational education curricula (i.e., integration of academic 
and vocational education, tech-prep) may contribute to making this debate obsolete. The creation of 
tech-prep programs may demand a new breed of vocational teacher. At the secondary level, there 
may be a need for teachers who have competence in a broad occupational field instead of a specific 
job and have greater facility at teaching the basics (mathematics, science, and English) as they apply 
to occupations. At the postsecondary level, teachers participating in tech-prep programs may need to 
meet the demands of better prepared high school students who have achieved higher levels of 
academic studies. A similar situation may exist for teachers who serve on professional teams that 
integrate academic and vocational education. This change may result in a need for vocational teachers 
who are themselves better prepared in mathematics, science, and English. Occupational experience, 
quality, and technical level may become increasingly critical. As vocational education evolves, the 
evolutionary nature of teacher qualifications may be a most meaningful area of study. 

Demand Issues 

In general, demand for teachers is a function of the number of students to be taught, class 
size, and teacher availability and attrition. However, with regard to vocational teachers, actual 
availability to teach can often be linked to the range of potential job opportunities in business and 
industry that are available to these teachers. Persons who can earn more money in business or 
industry than in teaching must do a great deal of soul-searching when it is time to make a job 
decision. Other factors certainly affect the demand for teachers. These are included in questions 
related to the area of teacher demand, provided below: 

L To what extent is there a demand for teachers at the postsecondary and secondary 
levels in various vocational teaching areas? 
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2. To what extent are persons available to teach at the secondary and postsecondary 
levels in various vocational teaching areas? 

3. Does current and future vocational teacher supply meet current and projected demand? 

4. Is vocational teacher demand shifting from the secondary to the postsecondary level, 
from various teaching areas to other teaching areas, and from a demand for fewer 
nondegreed to more degreed teachers? 

5. To what extent do certification requirements affect vocational teacher qualifications 
and thus affect demand? 

DATA SOURCES 

A review of research and literature related to vocational teacher preparation, qualifications, 
and demand indicates the availability of several potential data sources. Presented below are highlights 
of some of the more recent studies related to vocational teachers and teaching. 

Sources Related to Preparation 

Pratzner (1987) conducted a comprehensive national survey of vocational teacher preservice 
and inservice preparation. The survey focused on both preservice preparation programs and 
beginning teachers* perceptions of these programs. Pratzner's report includes extensive information 
about teacher preparation and qualifications. For example, it was determined that vocational teacher 
preparation programs gave little attention to preparation for teaching basic skills and special student 
populations. 

Another data source is an American Association of Colleges of Teacher Education (AACTE) 
teacher education survey (AACTE, 1990a). It includes enrollment in schools of teacher education by 
academic field and race/ethnicity. It was reported that as of Fall 1990, 4,128 students were enrolled 
in vocational teacher preparation. Enrollment distribution included: white, 75 percent; black, 15 
percent; Hispanic, 2 percent; and other, 4 percent. It should be noted that the AACTE has conducted 
a number of surveys related to teacher education which may have implications for vocational teacher 
preparation (e.g., AACTE, 1990b). 

A National Database on Vocational Teacher Education, a study conducted by the National 
Center for Research in Vocational Education (NCRVE), University of California-Berkeley (Lynch, 
1990), provides comprehensive information about the vocational teacher education environment, the 
vocational teacher education professorate, and the vocational teacher education curriculum within a 
context of reform. Another NCRVE study (Finch, Schmidt, Oliver, and Yu, 1991) focused on an 
analysis of vocational teacher education graduates* university transcripts. This study provides a 
detailed view of the academic, technical, and professional studies completed by vocational teacher 
education students in six vocational teaching areas. 

Sources Related to Qualifications 

Although most studies give only passing concern to vocational teacher qualifications, there are 
several investigations that may serve as meaningful source documents for this area. For example, the 
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Pratzner (1987) study results include a great deal of information related to vocational teacher 
qualifications. In their recent report related to a comprehensive study of vocational teaching, Heath- 
Camp and Camp (1991) provide the results of a national survey of new or beginning vocational 
teachers. The report includes data related to areas such as gender, race, marital status, subjects and 
grade levels taught, job satisfaction, and income. This information is particularly useful with regard 
to persons from nontraditional backgrounds (e.g., not having degrees in vocational teaching) who are 
entering the vocational teaching field. 

For some time, the National Center for Education Statistics (NCES) has conducted a Schools 
and Staffing Survey that may provide useful information about vocational teachers and teaching. The 
survey results include data related to vocational teacher demographic characteristics and educational 
backgrounds, and information by teaching area. The NCES is continually improving this 
comprehensive data base and sharpening its focus on vocational education teachers. 

Several studies focusing on vocational teacher certification can be logically linked to the 
qualifications area. Duenk's (1989) study of certification requirements for trade and industrial 
teachers provides a meaningful view of a teacher group that has traditionally taught without the 
bachelor's degree. Data gathered from 53 states and territories reveal that only two states require the 
bachelor's degree for initial certification. Ten states require the bachelor's degree for full 
certification. A parallel study in the health occupations teaching area (Chappelka, 1989) reveals 
similar certification information. Unfortunately, there does not appear to be a document that provides 
comprehensive details about certification requirements for all vocational teaching areas on a state-by- 
state basis. 

Sources Related to Demand 

Information related to vocational teacher supply and demand appears to be at least partially 
available. Such information is gathered via separate surveys conducted by persons representing 
various vocational teaching areas; therefore, comparability of information across teaching areas does 
not appear to be possible. 

A national study of teacher supply and demand in agricultural education has been conducted 
annually for the past 25 years. The most recent survey (Camp and Oliver, 1990) provides 
information on a state-by-state basis about agriculture teacher positions available and agriculture 
teachers prepared by teacher education institutions. Trend data from 1965 to the present are also^ 
provided. 

For a number of years, Industrial Education Magazine has conducted surveys of trade and 
industrial and technology education (formerly industrial arts) teacher supply and demand. Both the 
need for and shortage of teachers are reported on a state-by-state basis. A recent technology 
education teacher survey (Miller, 1990) includes information about the number of teachers currently 
employed in each state and the number of teachers produced by colleges and universities. With 
regard to the trade and industrial teacher survey (Greenan, 1990), demand information is presented by 
specific teaching area (e.g., auto mechanics, electronics, carpentry, and drafting). 

In the business education area, Stocker (1989) conducted the sixth in a series of studies 
focusing on business education in the United States. The study included information about business 
teacher supply and demand as perceived by business education departments in universities. This 
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report contains useful trend information for business education since data from all surveys are 
included. 

RESEARCH CONCERNS AND HOW THEY MAY BE ADDRESSED 

There are several research concerns regarding vocational teacher preparation, qualifications, 
and demand. First, it is clear that no comprehensive data base exists which encompasses the area in a 
logical and functional manner. As Lynch (1988) indicates, "Data is needed about vocational teacher 
education: Its scope, administrative structure, design, curriculum, standards, faculty, students, 
and— perhaps, most importantly— its actual and perceived [in] effectiveness." Clearly, some time will 
pass before such a comprehensive data base is developed. However, it is recognized that such 
information is needed for national policy and accountability purposes (Hoachlander, 1989). 

Several additional concerns and potential constraints emerged during the preparation of this 
paper. These concerns are listed below, as they have to some extent been discussed earlier in the 
paper. 

1. Teacher mobility, which is to some degree a reflection of the range of job 
opportunities available to vocational teachers (e.g., secondary, postsecondary, adult, 
business, industry, and government). 

2. Variation in state teacher-certification requirements. 

3. Range in teacher qualifications (e.g., degreed versus nondegreed teachers, different 
technical content associated with various vocational teaching areas). 

4. Range of teaching settings (e.g., comprehensive high school versus separate vocational 
school or center versus community or technical college). 

5. Heterogeneous nature of vocational teachers (e.g., beginning teachers may be younger 
or older and may be in their first, second, or third careers). 

6. Limited knowledge about part-time vocational teachers and how this group may be 
best served. 

In terms of dealing with concerns in the research process, several suggestions can be made. 
First, a close look must be taken at teachers in both secondary and postsecondary institutions. It is 
particularly important to assess the way teachers in these settings interact with each other. This will 
assist in capturing information about teacher linkages across institutions in relation to activities such 
as tech-prep programs. 

A second suggestion is to examine existing relationships among academic and vocational 
teachers. For example, do academic and vocational teachers work together in the education of all 
students? Do they share a common vision? Do they work as members of teaching teams? Do they 
collaborate on student assignments? This task should be accomplished both within and across 
institutions (e.g., secondary vocational centers and feeder schools). The hope is that such an 
approach will capture the dynamics of teachers working together to integrate academic and vocational 
education. 

247 



ERLC 



The third suggestion is that consideration be given to concurrently gathering teacher, student, 
school, and community information. This could be accomplished by employing a case-study 
methodology and using communities^and/or attendance areas as analysis units. Finally, although data 
bases in the area of vocational teachers and teaching are lacking, the sources provided in this paper, 
together with access to currently unpublished data sources (e.g., NCES, AACTE) may provide 
answers to at least some of the assessment questions raised. 



248 



REFERENCES 



American Association of Colleges for Teacher Education (1990a), Teacher Education Enrollment 
Survey. Washington, D.C.: AACTE Publications, Fall 1990. 

American Association of Colleges for Teacher Education (1990b), Teacher Education Policy in the 
States. Washington, D.C.: AACTE Publications. 

Berger, L. E., W. Feinberg, J. Pagano, and J.A. Witson, Preparing Teachers as Professionals. New 
York: Teachers College Press, 1989. 

Boyer, E. L., High School. New York: Harper and Row, 1983. 

Camp, W. G. and J. D. Oliver, A National Study of the Supply and Demand for Teachers of 

Agricultural Education in 1989. Blacksburg, Va.: Division of Vocational and Technical 
Education, Virginia Polytechnic Institute and State University, 1990. 

Carnegie Forum on Education and the Economy, A Nation Prepared: Teachers for the 21st Century. 
New York: Carnegie Corporation, 1986. 

Chappalka, C, Health Occupations Education Certification Requirements in the United States and 
Territories. Blacksburg, Va.: Division of Vocational and Technical Education, Virginia 
Polytechnic Institute and State University, 1989. 

Committee on Economic Development, Investing in Our Children. New York: Author, 1985. 

Diegmueller, K., "2 Oregon Colleges May Abolish Teacher Preparation," Education Week 5, 
February 20, 1991. 

Duenk, L. G», Trade and Industrial Education Certification Requirements in the United States and 
Territories. Blacksburg, Va.: Division of Vocational and Technical Education, Virginia 
Polytechnic Institute and State University. 1989. 

Finch, C. R., "Questions Raised by the Reports," Vocational Education Journal 62(4), 1987, pp. 29- 
30. 

Finch, C. R., "Challenges Facing Vocational Education Personnel Development," Southern Journal of 
Occupational Education 4(1), 1990, pp. 1-13. 

Finch, C. R., B. J. Schmidt, J. D. Oliver, and K. Yu, "University Studies as Preparation to Teach 
Academic Content in Vocational Education Classes," paper presented at the American 
Educational Research Association Meeting, Chicago, III., April 1991. 

Goodlad, J. A., A Place Called School. New York, N.Y.: McGraw-Hill, 1984. 

Greenan, J. P., "T&I Teachers: Supply and Demand," Industrial Education 79(1), 1990, pp. 13-15. 



249 

238 



Griggs, M.B M Research Priorities and Goals for Vocational Education Personnel Development. 
Berkeley, Calif.: National Center for Research in Vocational Education, University of 
California, 1990. 

Heath-Camp, B. and W.G. Camp, "A National Survey of the Experiences of Beginning Vocational 
Teachers," paper presented at the American Educational Research Association Meeting, 
Chicago, 111., April 1991. 

Hoachlander, E. G., National Data Needs for Vocational Education. Berkeley, Calif.: National 
Center for Research in Vocational Education, University of California, 1989. 

Holmes Group, Tomorrow's Teachers: A Report of the Holmes Group. East Lansing, Mich. 
Author, 1986. 

Hughes, R. P., "The Holmes Group Report," Vocational Education Journal 62(1), 1987, p. 29. 

Lee, J. S., "Coping With Reform," Vocational Education Journal 62(1), 1987, p. 30. 

Lynch, R. L., Using the Nominal Group Technique to Determine Priorities for Vocational Education 
Research. Blacksburg, Va.: Division of Vocational and Technical Education, Virginia 
Polytechnic Institute and State University, July 1987. 

Lynch, R. L., "Influencing Vocational Teacher Education Policy Through Research," in Beyond the 
Debate: Perspectives on the Preparation of Vocational Education Teachers. Macomb, HI.: 
Curriculum Publications Clearinghouse, 1988. 

Lynch, R. L., A National Database on Vocational Teacher Education. Berkeley, Calif.: National 
Center for Research in Vocational Education, University of California, 1990. 

Lynch, R. L. and M. B. Griggs, Vocational Teacher Education: A Context for the Future. Berkeley, 
Calif.: National Center for Research in Vocational Education, University of California, 1989. 

Miller, R., "IA/TE Teachers: Supply and Demand," Industrial Education 79(3), 1990, pp. 9-12. 

National Commission on Excellence in Education, A Nation at Risk: The Imperative for Educational 
Reform. Washington, D.C: U.S, Government Printing Office, 1985. 

National Commission on Secondary Vocational Education, The Unfinished Agenda: The Role of 

Vocational Education in the High School. Columbus, Ohio: National Center for Research in 
Vocational Education, Ohio State University, 1985. 

Pratzner, F. C, Vocational Teacher Education: A Survey ofPreservice and lnservice Preparation. 
Columbus, Ohio: National Center for Research in Vocational Education, Ohio State 
University, 1987. 



250 



Smith, C.W., "The Emergence of Educational Reform," Vocational Educational Journal 62(4), 
1987, pp. 38-40. 

Stocker, H. R., "Business Education in the United States: 1987-88 NABTE Survey Results, NABTE 
Review 17, 1989, pp. 5-17. 



251 

240 

o 

ERJ.C 



Performance Standards, Accountability, 
and the Quality of Vocational Education 



Paul T. Hill* 



INTRODUCTION 

v This paper is an expanded version of a briefing presented to the National Assessment of 

v Vocational Education planning conference on March IS, 1991. The paper applies the results of a 
study of accountability in vocational education, conducted by RAND for the National Center for 
Research on Vocational Education (NCRVE). 

The project on which this paper is based started in early 1990 and will be completed in late 
1991 . The project focuses on the M to whom" aspects of accountability, trying to understand who 
needs evidence about the performance of vocational education and how they would use it. NCRVE 
sees the project as a companion to other activities that examined the "for what" of vocational 
education accountability, considering the measurement implications of the Perkins Act performance 
requirements on performance standards and measures. 

This paper's message is that the main focus of accountability in vocational education is local, 
and that the Perkins Act requirements for new performance measures and standards are meant to 
strengthen local accountability, not to weaken it or to create new channels for direct reporting to the 
federal government. This conclusion has two implications for the National Assessment's work on 
accountability. First, the National Assessment should assess the character and health of local 
vocational education accountability processes. Second, the assessments of the effects of the Perkins 
Act requirements should emphasize local consequences rather than the building of standard national 
performance data bases. 

This analysis focuses on secondary vocational education, with some excursions to consider 
issues peculiar to postsecondary education. It relies on the following definition of accountability, as 
drawn from the briefing at the planning conference: 

• Accountability is a relationship between a service provider and anyone who has a 
well-founded expectation to derive benefits from the service. In vocational education, 
accountability refers to those processes whereby people who expect to benefit can 

judge whether their needs and expectations are being met, and 
determine whether programs should be changed, sustained, or eliminated. 

• Accountability runs in several directions: 

upward, to funders and legislators; 



♦Paul Hill is Senior Social Scientist, The RAND Corporation. 
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laterally, to others who educate students; and 
downward, to employers and students. 

The remainder of this paper follows the structure of the briefing, and includes discussion of 
the project's goals and conclusions. 

The NCRVE Accountability Project 

The goals of the NCRVE accountability project are to 

• understand Perkins Act implications for state and local 
performance measurement and accountability, and 

• suggest means of implementing Perkins requirements in order to 

strengthen local accountability processes; 
promote program improvement; and 
provide quality assurance for top officials. 

Conclusions 



The project results support the seven conclusions listed below and elaborated upon in 
succeeding pages. 



1. The Perkins Act was meant to strengthen accountability at the state and local levels, 
not to nationalize the design, management, or evaluation of vocational education. 

2. The Perkins requirements reflect a concern that local accountability processes, though 
strong in some places, are weak in many central cities and in other places where 
education generally fails to meet community needs. 

3. Even in those places where local accountability processes are strong, there are 
important conflicts of interest among local constituencies that justify oversight by state 
and federal governments. 

4. Hard outcomes data, particularly scores on competency tests that reflect true labor 
market needs, can help compensate for the limitations of local accountability 
processes. 

5. But sole reliance on hard outcomes data can distort local programs and ultimately 
weaken the performance and credibility of vocational education. Hard outcomes data 
(and state or national data bases built from them) should be used to supplement, not 
supplant, local processes of consultation, monitoring, and managerial judgment. 

6. Vocational educators take a great risk when they abandon traditional monitoring and 
management processes in favor of " automated " accountability processes based on 
formal measures and cut-offs. 
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7. The National Assessment's research should include a project focused on improving 
local accountability processes. 



Conclusion 1. The Perkins Act meant to strengthen local accountability 

A first goal of the NCRVE/RAND project was to understand the motivations of the people 
who drafted and supported the Perkins Act performance standards and accountability requirements. 
Interviews with congressional staff members and lobbyists made it clear that no one intended to create 
a flow of information to Washington that would let Congress or anyone else micromanage vocational 
education. Nor did the drafters hope to obtain a master data base from which they could derive an 
unambiguous assessment of vocational education, either nationwide or in any state or locality. 

It was clear, however, the members of Congress and senior committee staff were worried 
about the equality of local programs and uncertain about whether they were being implemented with 
due concern for the needs of local students and employers. This concern was based on business 
leaders' complaints about the quality of high school graduates (including, but not specific to, 
graduates of vocational programs), and on congressional staff members' perceptions that some local 
managers had become bureaucratic, deadened by routine, and more concerned with protecting staff 
jobs than with meeting local labor market needs. These concerns were based on impressions and 
testimony; nobody knew exactly how serious they were. However, congressional friends of 
vocational education found themselves on the defensive, with no strong evidence to support their 
positions. 

The Perkins Act requirements were meant to reassure critics that the legislative supporters of 
vocational education were serious about quality, and that lax or self-indulgent program administration 
would not be tolerated. However, these legislators did not want to set themselves up as the remote 
ultimate judges of local processes. They wanted, instead, to create a framework of processes which 
would guarantee that local quality issues received due attention at the local and state levels. 

These concerns were similar to the initial motivations for the evaluation requirements in 
ESEA Title I (now Chapter 1). As Robert F. Kennedy, the original drafter of the Title I 
requirements explained, the purpose of data and evaluation requirements was to ensure that local 
parents and citizens would know how well they were being served by federal money. Armed with 
that information, local people could press for improvements when necessary. (Only later, when 
researchers sought to draw national estimates of the productivity of Title I, was the original local 
purpose of the evaluation requirements forgotten). 

The drafters of the Perkins requirements have had decades of experience with Title I/Chapter 
I, and did not intend to stimulate another futile search for a national bottom-line assessment of a 
program that by design is local in goals, services, and outcomes. To the contrary, they intended to 
set up processes that would make local program performance manifest to local stake-holders. 

Conclusion 2. Local processes vary in quality 

In some localities, the quality and relevance of vocational education programs are ensured 
through such processes as 

• employer advisory committees, 
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tracking employer needs through student placement, 



attracting students via job placement success, 



employer certification of the need for new courses, and 



eliminating or upgrading courses with low enrollment and/or poor placement 
records. 



Howevei 



many local processes have serious weaknesses, including 



"routine following" mentality among providers, 



lax tracking of student placement, 



low expectations for student skill attainment and job placement, 



little capacity to project future labor force needs, and 



inattention to needs of transient, low-income, or minority students. 



Conclusion 3. Conflicts of interest among local constituencies justify state and federal oversight 

In very simple communities with only a few employers it is possible for healthy accountability 
processes to work in everyone's interest. The range of employment opportunities is finite and 
generally understandable, and all employers can take part in the normal processes of consultation. In 
larger communities, however, only a fraction of employers can be directly consulted. Larger and 
longer-established employers are naturally more likely to be consulted than very small or new 
employers. In some places, this means that important employer interests— the small firms that are the 
normal employers of new high school graduates and the newer firms that may rely on unfamiliar 
technologies and offer unconventional career opportunities— may be poorly represented. 

Conflicts of interest among students and employers may also arise. Established employers 
may want students prepared for traditional low- and moderate-skilled jobs, which might block students 
from obtaining jobs with other employers, possibly in other localities. 

A final possible conflict of interest may arise between students and educators. As Oakes has 
shown, some school systems emphasize college preparatory education for all but the lowest-achieving 
students, and others regard vocational education as the only appropriate placement for low-income 
minority students. In those cases, vocational education can gain a reputation for being, in Oakes* 
words, a dumping ground for students with poor academic preparation. Educators can therefore 
adopt low expectations for students and offer them unchallenging coursework that does not prepare 
them for rewarding careers. A similar sequence of events appears to be developing for immigrant 
students who come to the United States either unschooled or years behind their U.S. -educated peers. 
Older immigrant students tend also to be assigned to less challenging courses that prepare them only 
for low-skilled, dead-end jobs. 
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These conflicts of interest justify oversight by state and federal governments. The state needs 
to protect its broader economic future against the narrow localism and short-time horizon that can 
dominate local accountability processes. The state and federal governments need to help 
disadvantaged groups that lack the leverage to protect themselves at the local level. 

Conclusion 4. Competency test scores and other hard outcomes data can help compensate for 
local process limitations 

Local actors 9 horizons can be expanded in several ways. Information about future economic 
trends can help employers anticipate their own needs. Information about the needs of broader 
regional, statewide, or national labor markets can also help educators and students understand 
opportunities that might not be evident locally. Lists of competencies required by advanced 
employers— and testing systems to help students and educators assess their performance in light of 
those needs— can create demands for program improvement. 

Competency-based systems such as those being built in Oklahoma can both raise standards 
and provide concrete guides to program improvement. Such systems derive from lists of needed 
competencies created by knowledgeable employers. When these lists are used as the basis for 
curriculum development, student testing, and certification, they both guide the vocational education 
policy and build employers 9 competence. They also provide external standards against which 
program quality and student access can be evaluated. 

Conclusion 5* Hard outcomes data should supplement, not supplant, processes of consultation 
and monitoring and the use of managerial judgment 

All educational outcomes are products of multiple causation. Test scores, graduation rates, 
dropout rates, and student placement results can reflect on the quality of educational services. 
However, they can also be strongly affected by other factors independent of the quality of training 
and instruction students have received. Average outcomes for any group of students are affected by 
the group's composition. Groups selected on the basis of high ability or superior prior experience are 
likely to have more positive outcomes than less elite groups. Student-specific outcome measures 
(e.g., change scores) are less sensitive to group composition, but they are difficult to relate to a 
particular educational program. Students who have many positive educational experiences over the 
period during which change is measured are likely to get better change scores than students who had 
mixed or negative experiences. Thus, for example, students experiencing a given vocational program 
may have different change scores, depending on the quality of education they receive outside 
vocational education. 

The outcomes of greatest interest in vocational education are also affected by local labor 
market conditions. Vocational education students' job placement rates, hours of work, income, and 
rates of career progression can be profoundly affected by the demand for particular kinds of labor, 
which is in turn determined by the current level and composition of local economic activity. 

These complications do not rule out the use of hard outcomes data. However, they do 
illustrate the many ways in which such data can foster misunderstanding of program performance. 
Outcomes data are indispensable, but only when they can be interpreted in light of supplementary 
information about student characteristics and prior educational preparation, the quality of educational 
programs that complement vocational education, and local labor market conditions. In light of such 
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contextual information, programs can be fairly assessed and performance deficiencies can be readily 
identified. 

Without contextual information, vocational educators are forced to do everything possible to 
raise their outcome scores. This promotes "gaming" to increase scores (e.g., by "creaming" the 
student population or finding ways to discount the outcomes of less able students). It also misleads 
policymakers, frequently causing them to castigate local providers whose programs are in fact 
performing reasonably well in light of local conditions. 

Conclusion 6. Vocational educators take a great risk when they abandon traditional monitoring 
and management processes in favor of "automated" accountability processes 
based on formal measures and cut-offs 

Testing and formal outcomes measurement are not cheap, but they are far less labor intensive 
than direct program monitoring and careful accreditation of institutions. During a period of fiscal 
stringency, many states are tempted to cut their staffs and abandon direct oversight in favor of 
automated systems of measurement. 

These systems can be major assets, both for state and national officials responsible for 
vocational education and for actors at the local level. Only such "hard" measures can provide clear 
comparative information about the priorities and average outcomes attained by different localities and 
programs. However, as suggested above, hard outcomes data alone can lead to false attributions of 
causality and can mislead policymakers. State and federal officials may overlook crucial local 
conditions and reward or punish programs whose outcomes are largely determined, for good or ill, by 
uncontrollable local circumstances. 

Assessment and reward systems that are insensitive to local conditions can distort local 
programing and practice. If local program administrators and instructors become preoccupied about 
how they will score on standard outcome measurements, they often start neglecting their local 
accountability processes in favor of "gaming" the centralized reward system. In the short run, this 
can lead to the abuses mentioned in the previous section. The long-run consequences can be even 
more negative: Local providers of vocational education can come to de-emphasize the processes 
whereby they tailor vocational education to the needs of local students and employers. As we have 
seen in other major federal education programs (e.g., ESEA Chapter I), this leads providers to 
separate and distinguish the services they provide from services funded by other sources. 

As federal- or state-funded services become more distinctive and auditable, they also do less 
to complement and strengthen the basic educational programs provided at the local level. This 
process ultimately reduces the productivity cf the educational system as a whole, because students* 
performance in any one area is affected by the quality of their preparation in all areas. 

Because job competencies, placement rates, and other hard outcomes of vocational education 
are joint products of vocational and general education, vocational education cannot afford to employ 
an accountability system that ultimately weakens overall local educational performance. Vocational 
education cannot produce positive outcomes if students' general education preparation is abysmal. In 
education, a falling tide lowers all boats. Local citizens, employers, and civic leaders faced with a 
poorly performing education system seldom make fine distinctions among programs. Similarly, 
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national business leaders and others who decry the poor level of student preparation are not able to 
disentangle vocational from other forms of education; they only know that the system is failing them. 

When faced with complaints about system failure, top-level policymakers (Congress, state 
legislatures, presidents, governors) demand accountability. However, they mean systems 
accountability, not separate auditing of the effects of different programs. If the response to the call 
for accountability is to further separate educational programs from one another, system performance 
will continue to decline, with increasingly negative consequences for vocational education's reputation 
and public support. 

The bottom line is that vocational education must become an integral part of the local 
educational system, and its accountability system must emphasize meeting local needs. Standardized 
outcome measures can strengthen the local accountability process if they give local parents, 
employers, and community leaders objective information about local program outcomes. But if they 
are used in ways that weaken local accountability, standardized outcome measures can ultimately 
impair local program performance and further damage the local and national reputation of vocational 
education. 

\ national data base of standard outcome measures can only accelerate the centralization of 
accountability in vocational education. Local providers will assume that such a data base is a 
precursor to a system of centrally administered rewards and punishments, and act accordingly. 
National policymakers may not want to micromanage local or state programs, but news accounts 
drawn from the national data base will drive them to do so. In general, a national data base does not 
solve the problem of accountability in vocational education, and it threatens the performance of local 
programs. It is an example of a tool that determines its own use. If we do not want the 
consequences of a national outcomes data base we must avoid building one. The recommendations in 
the next section show how the National Assessment can help construct and build support for an 
alternative. 

Conclusion 7. Recommendations to the National Assessment 

Given the intense interest in better measures of vocational education outcomes, it is safe to 
assume that the National Assessment will monitor and assess the most advanced development efforts. 
The theme of the author's recommendations is that the National Assessment should also assess the 
status of the processes, particularly at the local level, whereby accountability takes place. The 
recommendation *s for a set of projects designed to: 

• Develop a national picture of the status of local accountability processes. This could 
be accomplished via 

A representative survey of school systems and postsecondary institutions; and 

Intensive case studies of a subset of systems and institutions, focusing on the 
character of local accountability processes and their effects on vocational 
education programs. 

• Study state efforts to strengthen links between local providers, employers, and 
parents. 
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• Characterize strong vs. weak local accountability processes and strong vs. weak state 
efforts to promote those processes. 

• Convene a conference to discuss 

How states and the federal government can identify strong and weak local 
accountability processes; 

What the federal government can do to help states monitor and assist local 
processes; and 

How state and federal data mandates can support, not detract from, local 
accountability. 

Participants at the proposed conference should include vocational educators, state legislators, and 
representatives of state and local business groups which have made significant investments of time and 
money in state and local education reform efforts. Business association representatives (e.g., the 
Chamber of Commerce) should take a back seat in this discussion. The purpose is to understand local 
business views of accountability and performance, not nationally homogenized views that encompass 
multiple agendas. 

This proposal is for a problem-solving project, not a data collection and analysis effort using 
standard methods. It is expensive because it requires significant case study work at the state and local 
levels, and it is labor-intensive because National Assessment staff must ultimately draw their 
conclusions through open consultation. It also puts the assessment staff in the middle of local 
relationships that may be emotionally charged and controversial. The compensating advantage is that 
the National Assessment will thereby help to emphasize the inherently local nature of accountability 
and create a local perspective which ensures that new outcome measures do not lead to a destructively 
centralized accountability system. 
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Integrating Academic and Vocational Education: 
Guidelines for Assessing a Fuzzy Reform 



Cathy Stasz and W. Norton Grubb* 



The 1990 Amendments to the Carl D, Perkins Vocational Education Act of 1984 require the 
National Assessment of Vocational Education to evaluate "the extent and success of integration of 
academic and vocational education." While research efforts by the National Center for Research in 
Vocational Education (NCRVE) 1 and others (e.g., Adelman, 1989) reveal much about current efforts 
to integrate academic and vocational education, it is still a fledgling reform, fuzzy in its intentions and 
methods. Integration means different things to different people and reform efforts vary widely, from 
fairly simple course changes to efforts that effectively restructure the high school. The National 
Assessment's challenge, then, is to conduct a credible evaluation of the potpourri of reforms that call 
themselves "integration." 

This paper, written to support the National Assessment's charter, has three goals: 

• To examine the principle themes and research issues related to integration of academic 
and vocational education. 

• To identify available data on integration and to suggest data gaps the National 
Assessment research might fill, 

• To address problems the National Assessment's evaluation of integration might 
encounter. 

In the first section of this paper, we provide some background on the integration "movement" 
by examining why this reform has caught the interest of policymakers, educators, and academics. We 
then discuss briefly what states and schools have been doing to integrate vocational and academic 
education, and we advance a more coherent definition of integration on which to base current and 
future research. Section two presents short- and long-term research questions for the National 
Assessment agenda, and discusses the availability of data and the data needs for answering these 
questions, in both secondary and postsecondary institutions. We also outline specific research studies 



*The authors are site directors of the National Center for Research in Vocational Education 
(NCRVE), the University of California, Berkeley. This paper represents their opinions only; it has not 
been reviewed by NCRVE or RAND (the subcontractor) and is not an official publication of either 
institution. 

'See Grubb et al. (1991), Bodilly et al. (forthcoming), Beck (1991), Plihal (1990), and forthcoming 
work by Grubb and Kraskouskas on postsecondary forms of integration; Jerry Pepple on applied 
academics courses; George Copa and Robert Beck on the history and philosophy of integration; and June 
Schmidt on teacher training. 
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the National Assessment might carry out, and address problems the National Assessment might 
encounter. Finally, we present conclusions and recommendations. 

WHAT'S ALL THE FUSS ABOUT INTEGRATION? 

Integration has captured the interest of educators, employers, and academics, each of whom 
sees it as a potential solution to specific problems. Critics of vocational education point to lowering 
enrollments, which some interpret as the result of a narrow vocational ism— an emphasis on job- 
specific skills— that is no longer relevant to the new workplace. Lower enrollments also reflect 
increased academic requirements for high school graduation, since vocational courses are electives not 
required for graduation. For this group, integration can help in several ways. An infusion of 
academic content in vocational courses can increase their rigor enough for them to be counted as 
required courses and can help prepare students more broadly for employment where requirements are 
constantly changing. In Ohio, for example, new "applied math" courses are team-taught by 
vocational and academic teachers, and students receive mathematics credit for the courses. The 
introduction of applied math, applied communications, and other similar courses reduces time devoted 
to the specific skill training that previously dominated the vocational program, and aims to provide 
students with a broader education. 

Employers have claimed that new job entrants lack basic skills or need different skills 
(problemsolving, teamwork, ^communication) to succeed in the high-tech workplace of the future. For 
this group, integrated courses would give vocational students more basic math than they would have 
otherwise, thus increasing their basic skills. Applied communications and other integrated English 
courses frequently include a variety of communication skills, such as writing letters or dealing with 
clients and customers. The hope is that integration will move the curriculum in ways that advance 
these "generic" skills (Stasz et al., 1990) as well as improve basic academic skills. 

In the decade since A Nation at Risk castigated the public schools, parents, educators, and a 
flurry of reports have echoed employers* concerns about low basic skills and lack of "higher-order 
thinking skills" (Resnick, 1987a). As a result, many school reformers seek to make academic 
learning more meaningful for all students and to prepare them better for the world of work. 
Although the term "integration" is not typically applied to reforms in academic education, new 
curricula and theoretical approaches have the flavor of making academic instruction more vocational, 
or at least more applied. For example, Resnick's (1987b) analysis of how school learning differs 
from out-of-school learning— in the workplace or throughout one's life— concludes that schooling 
should "encompass more of the features of successful out-of-school functioning," such as working in 
teams and using tools to solve real-life problems. Cognitive science researchers propose models of 
"cognitive apprenticeship" which draw heavily from studies of traditional vocational apprenticeship 
training (e.g., Lave, 1977) and recommend "situating" learning in contexts that reflect how a skill 
will be used (e.g., Collins, Brown, and Newman, 1989). 

With the advent of the 1990 Perkins Act, policymakers have joined the believers by targeting 
integration as a major vocational education program improvement that federal dollars will support. 
The Act requires that every program supported by federal funds "integrate academic and vocational 
education in such programs through coherent sequences of courses so that students achieve both 
academic and occupational competencies" (Section 235). Federal legislation, therefore, provides both 
the resources for integration and the pressure to integrate. However, it is crucial to point out that the 
various supporters of integrating academic and vocational education (i.e., policymakers, business 

262 



250 



representatives, reformers of vocational education, and critics of academic teaching) all have different 
reasons for adopting integration. As a result, integration is not a clear and unambiguous change, but 
instead is a fuzzy reform that comes in many guises. 

The most comprehensive survey of integration reforms to date (Grubb et a!., 1991) identifies 
eight different models, each with several variants, which serve many different goals. (These eight 
models are briefly described in table 1.) As a result, integration can aim to improve either basic 
skills or higher-order thinking skills. It can aspire to change the vocational program, concentrating 
on vocational students, or it can reshape the academic program and encompass all students. It can 
result in changing the content of a single course, or can restructure the way students learn in all 
courses. 

The way reformers define their goals affects what they do and how they measure their 
success. For example, if the goal is to improve higher-order thinking skills, the reform will probably 
focus on curriculum changes and measure students' inferential and problem-solving abilities. If die 
goal is to improve vocational students* job opportunities, the reform might reorganize vocational 
offerings in schools and forge better links to die business community, and then measure success in 
obtaining jobs. If the goal is to develop a broader version of vocational education, one less tied to 
specific job skills, preparation for postsecondary education or employment might be the measure of 
success. 

Research conducted to date suggests that the more ambitious attempts at integration include at 
least three elements. The first of these is curricular "alignment," which modifies both academic and 
vocational courses so that academic courses include more vocationally relevant material and 
vocational courses include more academic or basic content. In the most effective cases, academic and 
vocational teachers work together to achieve "horizontal - alignment, where coordinated courses are 
offered at the same time. Vertical alignment involves a coherent sequence of vocational and academic 
courses over time, rather than (or in addition to) coordination across courses at the same time (Grubb 
et al., 1991). 

Ambitious reforms are also characterized by changes in pedagogy. For example, Ohio has 
adopted "applied academics" curricula statewide, with vocational and academic teachers often teaching 
as a team. These curricula, as well as some popular off-the-shelf courses (e.g., Principles of 
Technology) feature an approach to learning math, physics, and communication skills that seems 
particularly suited to the learning needs of vocational students. With this approach, projects and lab 
activities are more frequently used to motivate learning, "academic" instruction is contextual ized by 
its use in occupational settings, and both students and teachers are more active in their approach to 
learning. 2 



*In the vocabulary developed in Grubb et al. (forthcoming), the pedagogical reforms in efforts at 
integration usually move from the conventional "skills and drills" methods common in the high school 
to very different methods we label "meaning-making," 
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Finally, many schools attempting integration have changed their organizational structures in 
significant ways. Some have abandoned the traditional department structure in favor of broad 
occupational clusters encompassing a variety of related occupations (e.g., transportation or health 
occupations) which house both vocational and academic teachers. Others, such as the "academy" 
model, are organized as schools-within-a-school. Still others "block" classes to permit longer time 
periods for vocational labs. 

While some states and schools have attempted ambitious reforms for integrating academic and 
vocational education, most have opted for fewer changes. By far, the most pervasive way to integrate 
involves adopting applied academics courses "off the shelf," especially principles of technology, 
applied math, and applied communication. Even so, this can involve a major effort on the part of 
teachers, particularly where they choose to develop their own curriculum materials or must 
extensively revise off-the-shelf materials. It is also important to see curriculum integration as a 
dynamic process in which schools start with simple changes and then move to more complex reforms. 
In this vision, applied academics courses represent a reasonable first step toward broader reforms. 

The Integration Dilemma 

At this point, policymakers, practitioners and researchers (including the National Assessment) 
face a dilemma. State policymakers must articulate plans for integration in order to receive funds. 
On the one hand, this provides both the incentive to reform and the resources to do it. On the other 
hand, the law is not at all specific about what counts as integration, beyond mentioning "a coherent 
sequence of courses so that students achieve both academic and vocational competencies." The 
integration "mandate" is further confused by other requirements which focus funding on special needs 
students. 

Where integration has not yet been attempted, practitioners may be searching for models and 
approaches that comply with the legislation but that also meet local needs and concerns. They often 
do this in an atmosphere where other local or state reforms may compete with their goals for 
integration. For example, state graduation requirements may determine whether or not applied 
academics courses can be counted as English or math credits. Certification requirements may 
determine which teachers can teach new courses. In some cases, state policy favoring job-specific 
skill training may collide with the intention in the Perkins Act to stimulate a broader form of 
vocational education. 

The problem of implementing top-down reforms in an area of education that requires local, 
bottom-up solutions is another part of the dilemma. Integration is essentially a curriculum reform that 
requires changes in course content and pedagogy. Mandates cannot create the elements that promote 
innovation in schools— local vision, leadership, sustained support, and financial and other resources. 

The dilemma for researchers is how to evaluate current and planned efforts when integration 
remains a reform for many purposes, with multiple goals and various expected outcomes. Perhaps 
more importantly, how can the National Assessment keep sight of the potential results of integration, 
not just the simple models or narrow reforms that seem to dominate practice at the moment? These 
potential outcomes include increased basic skills, increased theoretical and applied learning for all 
students, and increased breadth of postsecondary opportunities. These potential benefits of integration 
will take a long time to emerge, certainly longer than the current National Assessment 
charter. 
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RESEARCH ISSUES 



Obviously, the National Assessment must first focus its efforts on the short-term question: 
What are institutions doing in response to the new Perkins Act? In the following sections we propose 
a set of questions and a research strategy for addressing that issue in both secondary and 
postsecondary institutions. We also address general research issues and problems that affect the 
National Assessment agenda. 

In addition to the short-term question, the assessment studies should be sensitive to the long- 
term questions: Can we articulate effective models of integration that can be emulated by schools? Is 
integration a good idea? Eventually, researchers and practitioners must address the effectiveness of 
integration reforms. In the meantime, research can help articulate ambitious models that can 
potentially reform education in several ways (e.g., by increasing basic skills and academic content, by 
improving academic teaching, by increasing collaboration among teachers and enhancing their 
excitement about teaching and learning, by developing a more coherent curriculum, by preparing all 
students better for education and work, and by breaking down patterns of tracking and segregation). 

Secondary School Integration Issues 

At the secondary level, the National Assessment should be interested in two questions: 1) 
What state policies govern integration? 2) What are schools now doing to integrate? At the state 
level, existing data from NCRVE research (e.g., Grubb et al., 1991; Bodilly et ah, forthcoming; 
McDonnell and Zellman, forthcoming) show wide variation in state policies and few ambitious 
reforms. For example, preliminary results from NCRVE's 50-state survey show that many states 
have adopted the commercially available H applied academics'* curriculum as the major vehicle for 
integration. This reform essentially changes what is taught in individual courses and typically only 
affects vocational students. Ohio has mandated applied academic courses in its vocational program, 
but most teachers are developing their own curricula, and this is the only example of a statewide 
approach to integrating academic and vocational education. The most ambitious models, including 4- 
year vocational magnet schools and school-within-a-school models, are primarily the result of local 
reform efforts, not state initiatives. 

Under current regulations, states must submit draft plans by June 1991. The National 
Assessment could extend current knowledge by conducting a survey of state policy that asks how 
integration is addressed in the plan, both currently and in projected activities. The major research 
issue here is how ambitious the plan is in its attempts to integrate academic and vocational instruction. 
Does it opt for small changes or large ones? Does the plan include academic teachers and students, 
as well as those from the vocational side? Are there existing state policies or practices that hamper or 
limit integration, such as graduation requirements that threaten vocational education enrollments, 
teacher certification requirements, or course approval procedures that impede development of new 
courses? Does the plan restrict the concept of integration or promote certain approaches over others? 
How is the state using technical assistance and other forms of consultation to help districts formulate 
their own approaches to integration? 

With respect to how local schools respond to the integration mandate, we know from current 
research that approaches are enormously varied. Even so, they often lack a clear definition of or 
purpose for integration. Even in states with clear policies, such as Ohio, local implementation 
practices and the outcomes they produce differ. In our view, the only way to understand what 
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schools are doing in response to the mandate of the Perkins Act is to visit a large number of schools; 
that is, to undertake some carefully structured case studies of secondary schools? 



Secondary school case studies 

The case studies we envision should include districts with both comprehensive high schools 
and area vocational schools, as states vary in where and how secondary vocational education is 
provided. The sample can either be random or purposive, but in any case it should include some 
states and districts where interesting efforts are known to be taking place. In addition, because the 
Perkins Act directs federal aid to districts with high concentrations of special needs students, the 
sample should include sufficient numbers of such districts— including large urban school districts 
dominated by low-income and minority students— to see what form integration has taken. Intensive 
visits over several days are necessary to visit classrooms and interview the key actors, including 
principals, teachers, and students. 

Interviews and classroom observations should focus on answering several questions: What is 
the goal of integration in this school or district? What model is being implemented to reach that goal? 
What are the implementation plans or problems? What kinds of changes will integration mean for 
students, teachers, academic instruction, or guidance and counseling? What long- and short-term 
changes are expected? How will the effort be evaluated to determine if interim and long-term 
objectives are met? 

This case study of about 30 locations should be essentially descriptive, not evaluative. Below, 
we discuss several reasons why outcome evaluations would be premature, as well as methodologically 
unsound, in the context of the National Assessment. 

Postsecondary Integration Issues 

At the postsecondary level, the National Assessment will be interested in state policies that 
affect integration and in institutional responses to integration as a means of program improvement. 
Much less is currently known about state policy and local implementation of integration reforms at the 
postsecondary level than at the secondary level, partly because integration has been a much less 
urgent reform at the postsecondary level. (Indeed, while the requirement in the Perkins Act to 
integrate programs funded by federal resources extends to postsecondary institutions, this looks like a 
secondary reform that only by chance applies to postsecondary education.) 

Preliminary results from an NCRVE survey of about 300 community colleges and technical 
institutes (Grubb and Kraskouskas, forthcoming) indicate that the dominant form of integration 
consists of requiring vocational students to take general education classes. A few institutions are 
developing new curricula, such as vocational or technical mathematics and technical writing, and 
others are using "Writing Across the Curriculum" and "Reading Across the Curriculum" to change 
the content of vocational as well as academic courses. About one-fourth of those surveyed seemed to 



3 In our research for NCRVE, we found it impossible to rely on written or oral descriptions of what 
schools are doing; for more information on these efforts, see Gmbb et al., 1991, and Bodilly et al., 
forthcoming. Even brief site visits may fail to reveal the real intentions and effects of complex reforms. 

268 



be on the road to more ambitious reform, and were, for example, designing new interdisciplinary 
courses. With only spotty data currently available, the National Assessment should consider a survey 
of states to determine current policies for integration and methods for interpreting or enforcing the 
integration requirement. 

Postsecondary survey 

We suggest a survey of a random sample of about 20 states that devote a relatively high 
percentage of their Perkins funds to postsecondary education. (Some states provide so little aid to 
postsecondary institutions that it would be foolish to expect these resources to have any effect.) 
Within these states, locations with a mixture of community colleges, area vocational training schools, 
and technical institutes should be sampled. Rather than an intensive case study, we envision a lower 
level of effort, precisely because there seems to be less ferment at the postsecondary level. The 
survey of local practices in response to the new Perkins Act could be conducted over the telephone, 
with selected site visits to locations which show some activity in the area of integrated programs. 

The questions of interest overlap those aimed at secondary schools, but at a different level of 
detail. That is, the major focus should be on the state and institutional level, rather than on 
classrooms and teaching practices. Of particular interest here are the arguments and rationale offered 
for integration at the postsecondary level. In secondary schools, integration has been primarily a 
curriculum reform, justified by the need to better prepare students for work or postsecondary training 
and education opportunities. This argument may still be valid at the postsecondary level. For 
example, a recent analysis of health care technicians found that hospitals were satisfied with the 
technical qualifications of recent graduates but unhappy with their communications skills, suggesting 
that incorporating more academic content in health-tech programs would improve training (Hudis et 
aL, 1991). In addition, curriculum integration at the postsecondary level has the potential to join 
together parts of the community college that have been separated (e.g., the academic and the 
vocational sides or the vocational mission and the remedial role). However, there is much less clarity 
about the purposes of integration at the postsecondary level, and the issue of whether integration is 
desirable— or whether postsecondary programs should be quite specialized— should remain an open 
question. 

General Research Issues and Problems 

The National Assessment must consider several general research issues before determining a 
course of action. The first is the problem of timing. The initial year under the 1990 Perkins Act is 
1991-92. We anticipate that states and localities will spend a great deal of this year in initial planning 
and start-up activities; implementation of new programs may not occur until the 1992-93 school year. 
The earliest we might expect to see changes is 1993-94. Indeed, with a dynamic process, reform 
might reasonably take 5 or 6 years: year one would be involved in planning, year two would begin 
the process of change with the ninth grade, years three through five would complete the process of 
reform (assuming no special barriers or attempts to reformulate the change), and year six would be 
the first year a complete reform might be in place. However, the National Assessment must make its 
report to Congress in July 1994. The evaluation should first consider what kinds of changes might be 
expected within this time frame, rather than prematurely deciding that some efforts have failed. The 
focus might be on defining interim outcomes (e.g., replacing general science courses with applied 
academics) or future plans, rather than on M hard H student outcomes. 
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Evaluating the consequences of integration also poses some technical problems. Programs 
being implemented under the banner of integration vary widely in their purposes, scope, and stage of 
development. This situation makes it difficult to identify appropriate control or comparison groups, 
or to define comparable formative or summative outcome measures. Formal evaluations may not be 
warranted, given their high cost, the uncertainty about how to measure outcomes, the lack of control 
groups, and the timing problem (very few programs will be far enough along to expect any substantial 
outcomes by the time Congress reconsiders the Perkins Act). 

Given the importance of understanding the outcomes of integration, it is important to consider 
a variety of other options. The National Assessment might conduct several focused evaluations. For 
example, Ohio's statewide applied academics curriculum has been mandated for use in all vocational 
programs and is already being used in many schools. This creates a natural experiment where sites 
with and without this curriculum— in various stages of implementation— can be compared. (For 
example, ongoing NCRVE research in Ohio suggests that larger schools have more resources and 
more flexibility in implementing the changes associated with applied academics than do smaller 
schools.) 

A second target for focused evaluation is the off-ihe-shelf applied academics curricula 
developed by CORD/AIT, which serves as a main vehicle for integration in many states. An 
independent evaluation might examine the variations in implementing these courses, the teaching 
practices or other changes they affect, and how they serve local implementation goals. Ongoing 
NCRVE research indicates that vocational teachers often reject these materials as too "generic" or 
spend a considerable amount of time adapting them to be more applied to a particular occupational 
area. 

Third, longitudinal study of some long-standing programs might be considered. Some 
exemplary schools, identified in NCRVE work and other studies, are graduating students from 
integrated programs. These students could be followed up to examine their postsecondary options and 
choices and how integration affected them. 

Finally, a synthesis of existing evaluations might shed light on student outcomes. Two 
applicable studies are David Stern's research on the California Peninsula Academies (Stern et al., 
1988, 1989) and Robert Crain's NCRVE study of magnet schools in New York. Although previous 
research is not without its flaws, it provides a starting place for synthesizing what is known. 

CONCLUSIONS 

In our review of current efforts to integrate academic and vocational education (based 
primarily on ongoing research by NCRVE), we have drawn several conclusions. First, while 
integration embraces a varied range of reforms, most schools have so far opted for minor course-level 
changes. The education community needs to develop a clearer vision of what integration can 
potentially accomplish. This definition of integration would include at least three elements: vertical 
or horizontal alignment of courses; changes in pedagogy; and changes in organization (e.g., 
collaboration between academic and vocational teachers). Research should identify models that can be 
emulated by local institutions, based on local goals and needs. Single approaches or mandates will 
not suffice. 
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From NCRVE research and other sources, available baseline data (both case studies and 
surveys) describe various models of integration and current state policy at the secondary level. Much 
less information is available at the postsecondary level, but some studies are underway that can be 
built upon. New data collection at both levels is feasible (this paper has outlined studies for each 
level). Research should focus on current and future planning and implementation, since it is too early 
to expect much in the way of student outcomes. However, there are some opportunities for 
evaluating ongoing programs more formally and for synthesizing recent and current evaluation 
research. 

Several problems that may hamper attempts to evaluate the success of integration have been 
identified. First, there is a timing problem. New programs will not be in place before 1992; changes 
cannot be expected until 1993, if then. Technical problems arise in defining feasible comparison or 
control groups and outcome measures, especially when programs are so varied in practice* Finally, 
conceptual problems arise in addressing the long-term question of whether integration is a good idea. 
While current conceptions of integration may make sense at the secondary level, other conceptions, 
still to be articulated, may apply to postsecondary integration reforms. 
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Impact of Education Reform on Vocational Education 
Deborah C. Strickland* 



PERSPECTIVE 

Since 1983, educators have been exposed to a plethora of education reform initiatives and 
activities. By now, most of us have come to realize that the impact of reform on vocational education 
comes in a variety of packages. My own perspective on this topic has been influenced by two 
experiences. First, my 8-year involvement in the statewide evaluation of vocational education 
programs in Virginia, as required through the Perkins Act, allowed me to witness the flow of Perkins' 
legislative requirements through the state department of education and on to the local level of program 
delivery. At the same time, education reform initiatives were implemented at a variety of levels. For 
instance, the state department of education called for strengthening high school graduation 
requirements, localities revamped their curricula and opened magnet schools, and teacher education 
reform required colleges and universities to restructure their teacher training programs. In short, a 
partnership between education reform and vocations.* improvement quickly became a complicated 
matter. 

The second experience came through a research project (funded, in part, by the National Center 
for Research in Vocational Education [NCRVE]), in which my colleagues and I at Virginia Tech 
developed a longitudinal data base on high school and vocational enrollment patterns and a concurrent 
compendium of qualitative data or "journals** on the implementation of reform policies for each state. 
This study, the Vocational Enrollment Patterns Study (VEPS), was initiated to explore or "map" 
statewide trends in vocational participation vis-a-vis education reform, What we have learned so far 
in this study is that secondary school reform has isoved well beyond the single issue of high school 
graduation requirements and into the broader, more complex issues of delivery, choice, and 
outcomes. Moreover, for vocational education, being in the mainstream of these reforms is far more 
advantageous than being in the position ot playing "catch up." 

REFORM FRAMEWORK 

Public demand for education reform has resulted in implementation of new or revised education 
policies which appear to have implications for administration and delivery of vocational education 
programs. At the same time, the role of vocational education in the secondary school has come under 
increased scrutiny, and questions have been raised about the delivery and structure of programs. 
However, there are many states that have yet to realize the full impact of reforms or of the strategies 
put into place to restructure or revise vocational programs. In fact, the VEPS* initial emphasis was 
high school graduation requirements—an emphasis which quickly evolved into the broader picture of 
educational reform. For example: 



*The author is Deputy Director, Center for Survey Research, Division of Administration and 
Educational Services, Virginia Polytechnic Institute and State University. 
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1. The issue wis not simply increased graduation requirements, but more highly prescribed 
graduation requirements. While graduation requirements in some states did increase, other 
states simply made die existing number of requirements more prescriptive by designating 
the distribution of all required units. In either case, requirements for electives were often 
decreased, so that in order to complete a vocational program, students had to enroll for 
more credit hours than were required for graduation. According to many state vocational 
directors, this has presented a particular problem for at-risk students. 

2. In other states, high school graduation requirements alone were not the problem. Rather, 
the combination of these requirements with postsecondary (both 2-year and 4-year) entrance 
requirements created difficulty. Students who desired to continue their education beyond 
high school had to consider both high school graduation requirements and postsecondary 
entrance requirements in their decisions as to what courses to include in their curricula. 
For the most part, postsecondary entrance requirements precluded the choice of vocational 
education as an elective and the completion of a vocational program. 

3. State secondary school accreditation standards were another area of reform having an 
impact on vocational education. Changes in standards affected the availability and scope of 
programs within the schools which, in turn, affected student access to vocational education. 
In some cases, vocational education appeared to benefit. For example, standards requiring 
a predetermined scope of vocational programs within each school (rather than simply access 
to programs within the school system) appeared to facilitate students' ability to participate 
in vocational education. In other cases, accreditation standards increased the number or 
scope of educational programs to be offered in the schools, but reference to vocational 
education was left out altogether. Schools had to decide what to do with vocational 
education given their capability to satisfy the new standards. 

4. Teacher certification was another issue affecting vocational program delivery. 
Implementation of new certification requirements affected some states' staffing ability in 
vocational education. This was particularly true in the trade and industrial areas. In some 
cases, vocational education positions— and subsequently, programs— were lost. 

5. Parental choice policies have been introduced in a number of states. While still a fairly 
new reform initiative, one potential outcome is that as schools compete with each other to 
keep or attract students, retention of vocational education in the curriculum may be 
threatened. The impact of shifts in enrollments as students and parents choose among 
schools and, consequently, changes in program funding, quickly become issues of concern. 
Minnesota was one of the first states to raise this issue as a distinct problem for secondary 
vocational education. 

6. Equitable funding formulas among localities in a number of states (e.g., Kentucky, 
Tennessee, Ohio, and Alabama) have become both a reform and a legal issue. The issue in 
these cases often hinges on the availability of resources for academic positions and 
instruction. Consequently, the long-term impact of this issue on elective areas such as 
vocational education remains to be seen. 

In a recent study conducted by Chris Pipho for the Education Commission of the States (Pipho, 
1991), the progress of reform in all 50 states was reviewed. Pipho delineated some issues in 

276 



ERLC 



implementing education reform policies which have implications for vocational education as it tries 
either to initiate its own reform strategies or to survive the transitions within a general reform 
environment. Briefly, these issues point to problems of vision and leadership in the reform arena. 
Pipho's findings on these two points are summarized below: 

• The lack of a clear, coordinated vision for reform within states has resulted in inconsistent 
support and a lack of direction for school leaders otherwise willing to "change/ This 
schizophrenic approach to implementation at a variety of levels has resulted in reform 
activities which are more complicated and competitive than they are effective and are certainly 
more difficult to assess in terms of their impact on student learning and outcomes. As Pipho 
states, "...governors now say that the reforms are uncoordinated and beginning to trip over 
one another...." State leaders are beginning to question the success rates for different 
programs, how programs differ from one another, and "...who is coordinating the various 
reforms throughout the state" (p. 422). 

• Related to this lack of vision is the issue of leadership. The lack of coordination among 
reforms sends inconsistent messages to school leaders and inhibits their willingness to commit 
themselves to specific programs. Moreover, there is a lack of individuals who know how to 
lead in reform settings, which exacerbates the already complicated, uncoordinated nature of 
reform implementation. 

VOCATIONAL EDUCATION SCENARIO 

Not all vocational programs are created equal; some are in a better position than others to meet 
the demand for reform. Technology education is a good example, but often this program tends to 
straddle the fence in terms of its identification with vocational education. In the administration of 
VEPS surveys over the last 4 years, it became increasingly difficult to determine how to differentiate 
what was once called "industrial arts" from what is now supposed to be referred to as "technology 
education." Some states still retain the title— and presumably the programs— of "industrial arts." 
Others have incorporated industrial arts enrollments in the occupational program enrollments for trade 
and industrial education. Still others have made the change to "technology education" but also report 
enrollments in programs called "technical education" or "technology preparation." Most often, these 
latter programs are 2+2 programs articulated with postsecondary education. 

Another complication is the division of vocational education into various teaching areas, some of 
which have more or less acceptability in terms of being approved as alternatives for academic credit. 
Among states reporting the acceptance of vocational courses as alternatives, business math, 
agricultural science, and principles of technology were referenced most often. It should be 
recognized that these are not "hard core" vocational courses and that awarding credits through 
integrated vocational curricula appeared to be a limited option for students seeking to obtain 
vocational training while also satisfying their academic needs. 

The extent to which educators within vocational teaching areas work independently of each other 
affects the overall capacity of vocational education to initiate its own reforms or make the transitions 
required of all programs in the education reform environment. That "quality of separatism" Wirth 
(1973) described as characterizing vocational education's image in the secondary schools manifests 
itself within, as well as outside, vocational ranks. In this regard, efforts at integrating vocational and 
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academic curricula could prove to be an effective vehicle for enabling vocational education to 
participate comprehensively and jointly in the curricular transitions accompanying educational reform. 

Leadership in vocational education is an issue unto itself. It has demanded increased attention in 
recent literature and research efforts (see research conducted by Curtis Finch and his paper in this 
publication). The question continues to be: What is the capacity for vocational education to 
participate in the leadership of education reform in general and, subsequently, in vocational 
education's own reform? State vocational education directors are not always in the upper tier of state 
administration where they would be in a good position to coordinate or assert vocational education 
initiatives. Local administrators often have an array of responsibilities and are faced with asserting 
vocational leadership while also dealing with any number of issues or problems. Consequently, 
where general education reform has been complicated, vocational reform and the leadership required 
to implement reform become, at best, difficult. 

Another issue raised repeatedly by vocational directors in the VEPS surveys concerns the various 
roles vocational education is asked to serve. In a recent issue of the Kappan devoted to the status of 
vocational education, John Wirt delineated three ways vocational education could contribute to the 
goal of reforming education for all students: 1) by providing more of the context for learning 
academic subjects than is currently the case; 2) by providing more opportunities to use knowledge for 
practical purposes; and 3) by providing students with a broader range of personal, intellectual, and 
work-related skills than could be provided in traditional classrooms. This theme also runs through the 
writings of Lauren Resnick (1987) and others. (See Dan Inbar's forthcoming publication on "second 
chance" in education.) In addition to pointing out the many different ways people learn, both in and 
out of school, these writers have pressed the education system to provide multiple opportunities for 
students to obtain and build knowledge. Vocational education would do well to build on the 
momentum in these writings, but it has a long way to go in being widely accepted as a "second 
chance" vehicle. 

While vocational education is asked to continue to provide occupational skills to all students 
going into the work force, it appears to be increasingly drawn into serving the needs of special 
populations in particular (i.e., handicapped and otherwise disadvantaged students, displaced 
homemakers, adult learners, etc.). While this is a worthy role for vocational education, it sometimes 
appears to fragment an already struggling program. Combined with the fact that vocational education 
is an elective program for secondary students, this is a difficulty which lends itself to a program-wide 
identity crisis. 

A final level of complication comes through the interaction of secondary and postsecondary 
vocational education. When working with the Board of Regents staff of the University System of 
Georgia, I was made aware of a constant battle over who controlled career programs— the Board of 
Regents or the Department of Education. Although that was some years ago, I still see little 
cooperation or coordination within my current home state of Virginia or other states interviewed 
through the VEPS project. However, in the VEPS surveys, articulation and coordination of 
secondary and postsecondary programs were mentioned as critical factors in revitalizing vocational 
education. Combined with ongoing activities in tech-prep, these efforts could be the beginning of 
stronger, long-term collaborations between these two delivery systems. 
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IMPACT OF REFORM ON VOCATIONAL EDUCATION 



In the VEPS research on longitudinal enrollment patterns in vocational education and concurrent 
implementation of reform policies/strategies (Strickland, Elson, and Frantz, 1990), the 50 states and 
the District of Columbia were divided into eight different trend groups. They included four groups 
(19 states) characterized with increasing participation rates in vocational education and four groups 
(32 states) with decreasing participation rates. It was clear from the policy journals developed for 
each state (compiled from data collected through four VEPS survey administrations) that all states had 
experienced a variety of education reforms, most notably, changes in graduation requirements. In 
well over half the states, graduation requirements were cited as being the one factor having the most 
negative impact on vocational education participation. Factors cited as having the most positive 
influence on vocational education were academic/vocational curriculum integration and vocational 
program restructuring. 

After the states were grouped into the eight trend categories, the policy journals were reviewed 
for information common to states within each group. Based on this review, various policy activities 
or programmatic strategies which appeared to contribute to positive vocational participation trends 
were identified. These activities/strategies are summarized below: 

• Implementation of 2+2, advanced credit, and articulated programs (one vocational director 
noted that program participation improved through the "aggressive pursuit of 
secondary/posts econdary agreements"). 

• Introduction of exploratory, cluster, orientation, or career programs into the middle schools 
and in grades 9 and 10 in some cases where states* vocational education programs were 
limited to the upper grades. 

• Changes in accreditation standards which required vocational program availability within 
schools (or at least access to vocational education through local/area vocational centers, which 
had not been previously required). 

• Restructuring of vocational programs and implementation of more flexible scheduling 
patterns. 

• Initiatives in integrating vocational and academic curricula. 

• Recognition of vocational education in graduation requirements or of specific vocational 
education courses as alternatives for required academic credits. 

For states in the negative enrollment groups, the issue was not simply evidencing a different set 
of factors, but was rather a matter of not evidencing to the same extent the factors cited for the 
positive trend groups. However, state vocational directors in these groups did cite several issues 
which appeared to contribute to negative enrollment trends: 

• Exclusive emphasis on academic programs in graduation requirements and accreditation 
standards. 
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• The continued delivery of "traditional" vocational programs, multi-year programs, and 3- to 
4-hour block programs. (Travel time to regional centers was also a factor.) 

• Increased postsecondary entrance requirements. 

• Funding caps on personnel positions or units of vocational instruction. 

• Decline in high school population and states* economy. 

• Lack of secondary/postsecondary coordination, exploratory or orientation programs in middle 
grades, integrated curricula efforts, and vocational program innovations. 

• The image of vocational education as a program primarily for handicapped and disadvantaged 
students. 

(Findings from the VEPS should be regarded as descriptive.) 

The perspectives provided in this paper only begin to scratch the surface of education reform's 
impact on vocational education. By focusing our analysis of education reform's impact on changes in 
student vocational participation, we overlook other, equally important criteria by which to gauge the 
success of vocational programs. For example, Utah ties development and delivery of vocational 
programs to regional labor market demands. Sometimes, the programs needed to contribute to the 
local work force are not as popular as other programs which might be offered. In such instances, 
student participation may decline. However, to the extent that regional work force needs are served, 
the state's objectives for vocational education are met. 

THE NATIONAL ASSESSMENTS RESEARCH AGENDA 

Given the complex nature of reform activity among the states, strictly quantitative, "report card" 
assessments will do little to provide the insights needed about the effects of reform activities on 
vocational education. The National Assessment research agenda appears to need case studies, 
formative evaluations, and "how did we get here from there" analyses. The National Assessment 
should stress the incorporation or blending of qualitative research methods with more conventional 
quantitative analyses. However, given the time frame, we cannot dismiss the difficulty of initiating 
and completing qualitative assessments. The recommendation is to use focus groups of state leaders 
to determine the nature and extent of reform activities within states, and to use case studies at the 
local level to explore the top-down impact of education reform. Moreover, the hope is that the 
National Assessment will look at as closely at what does not work as at what does work. According 
to Pipho's study, there is a good deal of evidence that pilot programs often do not transfer well to 
other sites. Given the complicated and often uncoordinated framework of education reform, we may 
need to ferret out site-specific applications and ineffective strategies. We also may need to progress 
toward discovering effective approaches to reform and their subsequent impact on vocational 
education. 
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